00010000:00010000:2.0F:1787826471.574297:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:0.0F:1787826472.598213:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787826473.046206:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826473.046215:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:2.0:1787826473.046218:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826473.046222:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:2.0:1787826473.046225:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826473.046229:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:2.0:1787826473.046233:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826473.046236:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000080:00200008:3.0F:1787826473.202223:0:10289:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff8cd0e0513248), request_mask 2047, flags 0, foreign 0
00000080:00200000:3.0:1787826473.202239:0:10289:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8cd0e0513248),name=/
00000002:00010000:3.0:1787826473.202257:0:10289:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0
00010000:00010000:3.0:1787826473.202316:0:10289:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8cd0d27ee000 lock: ffff8cd0d28dc200/0x3298934de9c5a7c0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 10289 initiator: MDT0 lru_score: 0 lru_type: 0
00010000:00010000:3.0:1787826473.202327:0:10289:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8cd0d27ee000 lock: ffff8cd0d28dc200/0x3298934de9c5a7c0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 10289 initiator: MDT0 lru_score: 0 lru_type: 0
00010000:00010000:3.0:1787826473.202337:0:10289:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8cd0d27ee000 lock: ffff8cd0d28dc200/0x3298934de9c5a7c0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 10289 initiator: MDT0 lru_score: 0 lru_type: 0
00000100:00080000:3.0:1787826473.202363:0:10289:0:(client.c:1845:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != CONNECTING)  req@ffff8cd0c8ead680 x1874671626644480/t0(0) o101->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 576/1152 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/600/ffffffff rc 0/-1 job:'lsof.0' uid:0 gid:0 projid:0
00000100:00100000:3.0:1787826473.202374:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826473.622378:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826474.262205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826474.646240:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826475.286192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826475.670277:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826476.310202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826476.694536:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826477.142218:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787826462/real 1787826462]  req@ffff8cd0d48f70c0 x1874671626644352/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826477 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826477.142259:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626644352/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826477 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826477.142272:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787826462/real 1787826462]  req@ffff8cd0d48f5a40 x1874671626644096/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826477 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826477.142285:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626644096/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826477 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826477.142295:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787826462/real 1787826462]  req@ffff8cd0d48f6d00 x1874671626644224/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826477 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826477.142336:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626644224/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826477 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826477.142346:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787826462/real 1787826462]  req@ffff8cd0d48f61c0 x1874671626643968/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826477 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826477.142359:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626643968/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826477 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826477.142379:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826477.142415:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826477.142420:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626644352:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787826477.142427:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826477.142433:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826477.142437:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626644096:192.168.203.145@tcp:8:kworker.0
00000100:00080000:3.0:1787826477.142444:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826477.142450:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826477.142453:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626644224:192.168.203.145@tcp:38:kworker.0
00000100:00080000:2.0:1787826477.142458:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826477.142459:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787826477.142465:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787826477.142469:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826477.142474:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 353
00000100:00080000:2.0:1787826477.142484:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 10s
00000100:00080000:3.0:1787826477.142488:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00080000:2.0:1787826477.142493:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787826477.142495:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626643968:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787826477.142529:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626644608/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826477.142556:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826477.142560:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826477.142563:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826477.142567:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 353
00000100:00080000:2.0:1787826477.142571:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 10s
00000100:00080000:2.0:1787826477.142576:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787826477.142592:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626644608:192.168.203.145@tcp:250:kworker.0
00000008:00100000:2.0:1787826477.142595:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826477.142611:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626644736/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826477.142622:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826477.142625:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787826477.142627:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826477.142632:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 353
00000100:00080000:2.0:1787826477.142649:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 10s
00000100:00080000:2.0:1787826477.142655:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826477.142661:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826477.142673:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626644864/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826477.142681:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826477.142683:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826477.142685:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826477.142689:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 353
00000100:00080000:2.0:1787826477.142692:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 10s
00000100:00080000:2.0:1787826477.142697:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826477.142702:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826477.142714:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daad00 x1874671626644992/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826477.142724:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626644736:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826477.142772:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626644864:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826477.142815:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626644992:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826477.334251:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826477.718325:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826478.358405:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826478.742282:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826479.382211:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826479.766237:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000800:00000100:3.0:1787826480.278263:0:2347:0:(socklnd_cb.c:2254:ksocknal_find_timed_out_conn()) A connection with 12345-192.168.203.145@tcp (192.168.203.145:988) timed out; the network or node may be down.
00000100:00100000:3.0:1787826480.406199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826480.790306:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826481.430514:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826481.814770:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787826482.262237:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826482.262246:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:0.0:1787826482.262250:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826482.262252:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:0.0:1787826482.262255:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826482.262258:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:0.0:1787826482.262260:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826482.262263:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00100000:3.0:1787826482.454455:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826482.838217:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826483.478199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826483.864138:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826484.502315:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826484.886272:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826485.526399:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826485.910337:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826486.550292:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826486.934429:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0F:1787826487.382425:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826487.382450:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1787826487.382454:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826487.382457:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1787826487.382459:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826487.382463:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:1.0:1787826487.382465:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826487.382468:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00100000:3.0:1787826487.574240:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826487.958422:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826488.598203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826488.982269:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826489.622268:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826490.006355:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826490.646318:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826491.030393:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826491.670453:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826492.054941:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787826492.502273:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826492.502281:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:2.0:1787826492.502301:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826492.502304:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:2.0:1787826492.502307:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826492.502309:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:2.0:1787826492.502311:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826492.502314:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00100000:3.0:1787826492.694207:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826493.078236:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826493.718243:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826494.103122:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826494.742273:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826495.126583:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826495.766193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826496.152212:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826496.790211:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826497.174576:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787826497.622270:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826497.622279:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:0.0:1787826497.622282:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826497.622285:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:0.0:1787826497.622288:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787826497.622299:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787826477/real 1787826477]  req@ffff8cd0c3daad00 x1874671626644992/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826497 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826497.622313:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:0.0:1787826497.622318:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826497.622321:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00100000:3.0:1787826497.622343:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daad00 x1874671626644992/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826497 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826497.622355:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787826477/real 1787826477]  req@ffff8cd0c3daa580 x1874671626644736/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826497 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826497.622365:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626644736/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826497 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826497.622373:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787826477/real 1787826477]  req@ffff8cd0c3da92c0 x1874671626644864/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826497 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826497.622383:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626644864/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826497 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826497.622392:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787826477/real 1787826477]  req@ffff8cd0c3da9e00 x1874671626644608/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826497 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826497.622412:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626644608/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826497 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826497.622431:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826497.622443:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826497.622447:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626644992:192.168.203.145@tcp:38:kworker.0
00000100:00080000:3.0:1787826497.622459:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826497.622465:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00080000:0.0:1787826497.622470:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787826497.622470:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626644736:192.168.203.145@tcp:38:kworker.0
00000100:00080000:0.0:1787826497.622476:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: not pinging (in recovery or recovery disabled: CONNECTING)
00000100:00080000:3.0:1787826497.622478:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826497.622479:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826497.622484:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826497.622486:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826497.622490:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 368
00000100:00080000:3.0:1787826497.622496:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00080000:0.0:1787826497.622498:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 15s
00000100:00100000:3.0:1787826497.622502:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626644864:192.168.203.145@tcp:8:kworker.0
00000100:00080000:0.0:1787826497.622507:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00080000:3.0:1787826497.622508:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826497.622515:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826497.622519:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626644608:192.168.203.145@tcp:250:kworker.0
00000008:00100000:0.0:1787826497.622520:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826497.622551:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626645120/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826497.622565:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826497.622567:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787826497.622569:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826497.622572:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 368
00000100:00080000:0.0:1787826497.622575:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 15s
00000100:00080000:0.0:1787826497.622580:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787826497.622581:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626645120:192.168.203.145@tcp:38:kworker.0
00000008:00100000:0.0:1787826497.622585:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826497.622597:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626645248/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826497.622614:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826497.622617:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826497.622619:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826497.622623:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 368
00000100:00080000:0.0:1787826497.622628:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 15s
00000100:00080000:0.0:1787826497.622632:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826497.622637:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826497.622658:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626645376/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826497.622663:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626645248:192.168.203.145@tcp:8:kworker.0
00000100:00080000:0.0:1787826497.622679:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826497.622683:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787826497.622685:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826497.622694:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 368
00000100:00080000:0.0:1787826497.622700:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 15s
00000100:00100000:3.0:1787826497.622705:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626645376:192.168.203.145@tcp:38:kworker.0
00000100:00080000:0.0:1787826497.622706:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826497.622724:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626645504/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826497.622763:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626645504:192.168.203.145@tcp:250:kworker.0
00000400:00000100:2.0:1787826497.686424:0:2346:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1021 -> 192.168.203.145:988
00000400:00000100:2.0:1787826497.686494:0:2346:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:2.0:1787826497.686513:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787826497.686560:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787826497.686572:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787826497.686582:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:3.0:1787826497.686611:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826497/real 1787826497]  req@ffff8cd0c8e9d680 x1874671626645504/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826522 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826497.686637:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626645504/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826522 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826497.686650:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826497.686655:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826497.686659:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626645504:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826497.686688:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826497/real 1787826497]  req@ffff8cd0c8e9de00 x1874671626645376/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826522 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826497.686701:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626645376/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826522 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826497.686709:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826497.686713:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826497.686720:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626645376:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826497.686728:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826497/real 1787826497]  req@ffff8cd0c8e9e580 x1874671626645248/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826522 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826497.686741:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626645248/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826522 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826497.686749:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826497.686752:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826497.686756:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626645248:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826497.686762:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826497/real 1787826497]  req@ffff8cd0c8e9cf00 x1874671626645120/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826522 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826497.686775:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626645120/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826522 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826497.686783:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826497.686786:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826497.686789:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626645120:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826497.814209:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826498.198342:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826498.838179:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826499.222236:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826499.862143:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826500.246343:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826500.886171:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826501.270365:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826501.910268:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826502.295187:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787826502.742187:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826502.742197:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787826502.742201:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826502.742210:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 388
00000100:00080000:1.0:1787826502.742219:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826502.742224:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787826502.742253:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626645632/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826502.742274:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826502.742276:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826502.742277:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826502.742279:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 388
00000100:00080000:1.0:1787826502.742282:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826502.742284:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826502.742292:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826502.742317:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626645760/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826502.742328:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826502.742331:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787826502.742332:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826502.742336:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 388
00000100:00080000:1.0:1787826502.742340:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826502.742343:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826502.742348:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826502.742359:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626645888/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826502.742368:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826502.742372:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826502.742373:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826502.742378:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 388
00000100:00080000:1.0:1787826502.742381:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826502.742384:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826502.742389:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826502.742398:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626645632:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787826502.742402:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626646016/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826502.742484:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626645760:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826502.742516:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626645888:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826502.742543:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626646016:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826502.934203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826503.318271:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:2.0:1787826503.830906:0:2343:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1021 -> 192.168.203.145:988
00000400:00000100:2.0:1787826503.830944:0:2343:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:2.0:1787826503.830958:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787826503.830993:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787826503.831000:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787826503.831007:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:3.0:1787826503.831069:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826502/real 1787826503]  req@ffff8cd0d48f6d00 x1874671626646016/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826527 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826503.831122:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626646016/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826527 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826503.831134:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826503.831139:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826503.831143:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626646016:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826503.831150:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826502/real 1787826503]  req@ffff8cd0d48f52c0 x1874671626645632/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826527 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826503.831160:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626645632/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826527 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826503.831181:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826503.831184:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826503.831187:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626645632:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826503.831192:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826502/real 1787826503]  req@ffff8cd0d48f7c00 x1874671626645760/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826527 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826503.831203:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626645760/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826527 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826503.831210:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826503.831213:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826503.831215:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626645760:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826503.831220:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826502/real 1787826503]  req@ffff8cd0d48f61c0 x1874671626645888/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826527 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826503.831229:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626645888/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826527 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826503.831235:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826503.831237:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826503.831240:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626645888:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826503.958246:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826504.342324:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826504.982419:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826505.366301:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826506.006188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826506.390305:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826507.030232:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826507.414223:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787826507.862212:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826507.862221:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787826507.862225:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826507.862260:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 393
00000100:00080000:1.0:1787826507.862271:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826507.862279:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787826507.862315:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626646144/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826507.862346:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826507.862350:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826507.862352:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826507.862355:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 393
00000100:00080000:1.0:1787826507.862359:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826507.862363:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826507.862372:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826507.862386:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626646272/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826507.862395:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626646144:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787826507.862401:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826507.862405:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787826507.862407:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826507.862411:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 393
00000100:00080000:1.0:1787826507.862415:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826507.862419:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826507.862425:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826507.862435:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626646400/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826507.862458:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826507.862461:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826507.862463:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826507.862467:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 393
00000100:00080000:1.0:1787826507.862472:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826507.862477:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826507.862482:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826507.862495:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626646528/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826507.862502:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626646272:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826507.862537:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626646400:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826507.862562:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626646528:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826508.054236:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826508.438377:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826509.078204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826509.462227:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:0.0:1787826509.975165:0:2345:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1021 -> 192.168.203.145:988
00000400:00000100:0.0:1787826509.975233:0:2345:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:0.0:1787826509.975248:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:3.0:1787826509.975550:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826507/real 1787826509]  req@ffff8cd0d48f5a40 x1874671626646144/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826532 ref 2 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826509.975574:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626646144/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826532 ref 2 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826509.975585:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826509.975589:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826509.975593:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626646144:192.168.203.145@tcp:250:kworker.0
00000800:00000100:0.0:1787826509.975650:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:0.0:1787826509.975668:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:0.0:1787826509.975675:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:3.0:1787826509.975972:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826507/real 1787826509]  req@ffff8cd0d48f7840 x1874671626646528/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826532 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826509.975986:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626646528/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826532 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826509.975995:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826509.975998:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826509.976002:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626646528:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826509.976018:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826507/real 1787826509]  req@ffff8cd0d48f70c0 x1874671626646272/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826532 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826509.976031:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626646272/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826532 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826509.976038:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826509.976041:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826509.976051:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626646272:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826509.976057:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826507/real 1787826509]  req@ffff8cd0d48f43c0 x1874671626646400/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826532 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826509.976069:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626646400/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826532 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826509.976076:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826509.976079:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826509.976121:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626646400:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826510.102205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826510.486247:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826511.126196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826511.510267:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826512.150198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826512.534248:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787826512.982303:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826512.982320:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787826512.982325:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826512.982342:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 398
00000100:00080000:1.0:1787826512.982356:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826512.982374:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787826512.982434:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626646656/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826512.982462:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826512.982467:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826512.982469:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826512.982473:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 398
00000100:00080000:1.0:1787826512.982477:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826512.982482:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826512.982529:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826512.982546:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626646784/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826512.982558:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826512.982562:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787826512.982564:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826512.982568:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 398
00000100:00080000:1.0:1787826512.982574:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826512.982579:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826512.982585:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826512.982598:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626646912/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826512.982609:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826512.982612:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826512.982614:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826512.982618:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 398
00000100:00080000:1.0:1787826512.982622:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826512.982627:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826512.982632:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826512.982645:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626647040/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826512.982686:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626646656:192.168.203.145@tcp:250:kworker.0
00000800:00000100:3.0:1787826512.982747:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826512.982778:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826512/real 1787826512]  req@ffff8cd0d48f4b40 x1874671626646656/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826537 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826512.982834:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626646656/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826537 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826512.982846:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826512.982850:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826512.982854:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626646656:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826512.982862:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626646784:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826512.982881:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826512.982890:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826512/real 1787826512]  req@ffff8cd0d48f6580 x1874671626646784/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826537 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826512.982897:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626646784/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826537 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826512.982902:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826512.982906:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826512.982909:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626646784:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826512.982915:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626646912:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826512.982933:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826512.982944:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826512/real 1787826512]  req@ffff8cd0d48f7480 x1874671626646912/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826537 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826512.982955:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626646912/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826537 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826512.982972:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826512.982977:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826512.982980:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626646912:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826512.982985:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626647040:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826512.983007:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826512.983019:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826512/real 1787826512]  req@ffff8cd0d48f4000 x1874671626647040/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826537 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826512.983029:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626647040/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826537 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826512.983036:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826512.983039:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826512.983041:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626647040:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826513.174204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826513.558229:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826514.198282:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826514.582272:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826515.222199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826515.606245:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826516.246207:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826516.630289:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826517.270368:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826517.654245:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787826518.102207:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826518.102217:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787826518.102237:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826518.102247:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 403
00000100:00080000:1.0:1787826518.102256:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826518.102263:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787826518.102299:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626647168/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826518.102326:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826518.102330:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826518.102331:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826518.102335:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 403
00000100:00080000:1.0:1787826518.102338:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826518.102347:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826518.102357:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826518.102370:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626647296/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826518.102379:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626647168:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787826518.102383:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826518.102387:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787826518.102389:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826518.102393:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 403
00000100:00080000:1.0:1787826518.102396:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826518.102400:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826518.102407:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826518.102418:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626647424/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826518.102438:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826518.102441:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826518.102443:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826518.102447:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 403
00000100:00080000:1.0:1787826518.102452:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826518.102456:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826518.102462:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826518.102474:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626647552/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826518.102480:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626647296:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826518.102519:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626647424:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826518.102545:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626647552:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826518.294188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826518.678255:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:0.0:1787826519.192906:0:2344:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1021 -> 192.168.203.145:988
00000400:00000100:0.0:1787826519.192962:0:2344:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:0.0:1787826519.192979:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:0.0:1787826519.193027:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:0.0:1787826519.193036:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:0.0:1787826519.193042:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:3.0:1787826519.193912:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826518/real 1787826519]  req@ffff8cd0d48f4b40 x1874671626647552/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826543 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826519.193937:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626647552/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826543 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826519.193948:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826519.193953:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826519.193957:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626647552:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826519.193965:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826518/real 1787826519]  req@ffff8cd0d48f7480 x1874671626647296/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826543 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826519.193976:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626647296/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826543 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826519.193983:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826519.193986:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826519.193989:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626647296:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826519.193995:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826518/real 1787826519]  req@ffff8cd0d48f6580 x1874671626647424/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826543 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826519.194006:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626647424/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826543 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826519.194013:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826519.194016:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826519.194034:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626647424:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826519.194040:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826518/real 1787826519]  req@ffff8cd0d48f4000 x1874671626647168/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826543 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826519.194052:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626647168/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826543 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826519.194059:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826519.194062:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826519.194065:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626647168:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826519.318190:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826519.702239:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826520.342189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826520.726249:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826521.366227:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826521.750206:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826522.390190:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826522.774575:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787826523.222215:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826523.222225:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787826523.222229:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826523.222239:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 409
00000100:00080000:1.0:1787826523.222249:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826523.222255:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787826523.222297:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626647680/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826523.222323:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826523.222327:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826523.222328:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826523.222332:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 409
00000100:00080000:1.0:1787826523.222335:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826523.222339:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826523.222361:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826523.222373:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626647680:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787826523.222376:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626647808/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826523.222393:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826523.222396:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787826523.222398:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826523.222401:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 409
00000100:00080000:1.0:1787826523.222405:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826523.222409:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826523.222413:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826523.222423:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626647936/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826523.222432:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826523.222435:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826523.222436:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826523.222442:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 409
00000800:00000100:3.0:1787826523.222444:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787826523.222446:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826523.222451:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826523.222456:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826523.222465:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626648064/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826523.222503:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826523/real 1787826523]  req@ffff8cd0d48f43c0 x1874671626647680/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826548 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826523.222522:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626647680/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826548 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826523.222532:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826523.222535:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826523.222539:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626647680:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826523.222560:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626647808:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826523.222583:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826523.222604:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826523/real 1787826523]  req@ffff8cd0d48f70c0 x1874671626647808/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826548 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826523.222617:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626647808/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826548 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826523.222623:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826523.222625:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826523.222627:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626647808:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826523.222631:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626647936:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826523.222649:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826523.222660:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826523/real 1787826523]  req@ffff8cd0d48f7840 x1874671626647936/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826548 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826523.222670:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626647936/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826548 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826523.222677:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826523.222680:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826523.222683:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626647936:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826523.222689:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626648064:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826523.222707:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826523.222717:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826523/real 1787826523]  req@ffff8cd0d48f5a40 x1874671626648064/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826548 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826523.222728:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626648064/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826548 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826523.222742:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826523.222761:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826523.222765:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626648064:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826523.414191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826523.798267:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826524.438250:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826524.822346:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826525.462195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826525.846266:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826526.486238:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826526.870296:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826527.510843:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826527.894229:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787826528.342214:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826528.342224:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787826528.342228:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826528.342237:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 414
00000100:00080000:0.0:1787826528.342247:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826528.342254:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826528.342297:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626648192/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826528.342332:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826528.342335:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826528.342337:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826528.342340:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 414
00000100:00080000:0.0:1787826528.342343:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826528.342347:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826528.342356:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826528.342367:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626648320/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826528.342375:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626648192:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787826528.342380:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826528.342384:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787826528.342386:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826528.342390:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 414
00000100:00080000:0.0:1787826528.342393:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826528.342397:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826528.342401:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826528.342412:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626648448/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826528.342420:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826528.342423:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826528.342425:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826528.342429:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 414
00000100:00080000:0.0:1787826528.342515:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826528.342521:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826528.342526:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826528.342538:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626648576/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826528.342619:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626648320:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826528.342650:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626648448:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826528.342671:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626648576:192.168.203.145@tcp:38:kworker.0
00000400:00000100:2.0:1787826528.406372:0:2346:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1021 -> 192.168.203.145:988
00000400:00000100:2.0:1787826528.406420:0:2346:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:2.0:1787826528.406441:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787826528.406474:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787826528.406485:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787826528.406493:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:3.0:1787826528.406517:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826528/real 1787826528]  req@ffff8cd0c8e9fc00 x1874671626648320/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826553 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826528.406542:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626648320/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826553 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826528.406553:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826528.406557:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826528.406561:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626648320:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826528.406569:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826528/real 1787826528]  req@ffff8cd0c8e9cf00 x1874671626648448/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826553 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826528.406600:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626648448/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826553 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826528.406608:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826528.406611:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826528.406615:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626648448:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826528.406621:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826528/real 1787826528]  req@ffff8cd0c8e9e580 x1874671626648576/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826553 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826528.406632:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626648576/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826553 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826528.406639:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826528.406642:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826528.406645:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626648576:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826528.406652:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826528/real 1787826528]  req@ffff8cd0c8e9e940 x1874671626648192/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826553 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826528.406662:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626648192/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826553 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826528.406669:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826528.406672:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826528.406675:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626648192:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826528.534208:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826528.918754:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826529.558327:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826529.942238:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826530.582193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826530.966303:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826531.606208:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826531.990272:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826532.630208:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826533.014302:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787826533.462205:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826533.462216:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787826533.462219:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826533.462229:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 419
00000100:00080000:0.0:1787826533.462241:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826533.462248:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826533.462283:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626648704/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826533.462312:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826533.462315:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826533.462317:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826533.462321:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 419
00000100:00080000:0.0:1787826533.462325:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826533.462341:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826533.462352:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826533.462367:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626648704:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826533.462368:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626648832/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826533.462403:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826533.462407:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787826533.462409:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826533.462413:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 419
00000100:00080000:0.0:1787826533.462420:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826533.462424:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826533.462430:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787826533.462431:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826533.462447:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626648960/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826533.462458:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826533.462462:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826533.462463:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1787826533.462475:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826533/real 1787826533]  req@ffff8cd0c8e9de00 x1874671626648704/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826558 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826533.462482:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 419
00000100:00080000:0.0:1787826533.462490:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826533.462496:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787826533.462499:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626648704/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826558 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:0.0:1787826533.462502:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00080000:3.0:1787826533.462515:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:0.0:1787826533.462520:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626649088/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826533.462523:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826533.462531:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626648704:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826533.462574:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626648832:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826533.462598:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826533.462611:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826533/real 1787826533]  req@ffff8cd0c8e9d680 x1874671626648832/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826558 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826533.462623:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626648832/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826558 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826533.462630:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826533.462633:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826533.462636:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626648832:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826533.462648:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626648960:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826533.462671:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826533.462697:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826533/real 1787826533]  req@ffff8cd0c8e9ed00 x1874671626648960/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826558 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826533.462709:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626648960/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826558 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826533.462716:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826533.462719:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826533.462722:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626648960:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826533.462728:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626649088:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826533.462747:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826533.462757:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826533/real 1787826533]  req@ffff8cd0c8e9f840 x1874671626649088/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826558 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826533.462767:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626649088/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826558 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826533.462775:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826533.462778:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826533.462781:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626649088:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826533.654229:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826534.038237:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826534.678188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826535.063580:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826535.702197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826536.086259:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826536.726152:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826537.110311:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826537.750221:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826538.134213:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787826538.582582:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826538.582592:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787826538.582610:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826538.582619:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 424
00000100:00080000:0.0:1787826538.582628:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826538.582635:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826538.582678:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626649216/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826538.582703:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826538.582707:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826538.582709:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826538.582713:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 424
00000100:00080000:0.0:1787826538.582716:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826538.582720:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826538.582729:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826538.582747:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626649344/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826538.582748:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626649216:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787826538.582765:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826538.582769:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787826538.582771:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826538.582783:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 424
00000100:00080000:0.0:1787826538.582788:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826538.582792:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826538.582797:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826538.582809:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626649472/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787826538.582813:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787826538.582834:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826538.582838:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826538.582840:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826538.582844:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 424
00000100:00080000:0.0:1787826538.582849:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826538.582856:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787826538.582857:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826538/real 1787826538]  req@ffff8cd0c8e9c000 x1874671626649216/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826563 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:0.0:1787826538.582863:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826538.582879:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626649600/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826538.582882:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626649216/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826563 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826538.582897:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826538.582903:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826538.582909:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626649216:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826538.582933:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626649344:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826538.582954:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826538.582967:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826538/real 1787826538]  req@ffff8cd0c8e9f480 x1874671626649344/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826563 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826538.582980:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626649344/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826563 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826538.582988:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826538.582991:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826538.582994:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626649344:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826538.583000:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626649472:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826538.583029:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826538.583040:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826538/real 1787826538]  req@ffff8cd0c8e9cb40 x1874671626649472/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826563 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826538.583061:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626649472/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826563 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826538.583070:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826538.583073:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826538.583076:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626649472:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826538.583083:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626649600:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826538.583123:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826538.583134:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826538/real 1787826538]  req@ffff8cd0c8e9c3c0 x1874671626649600/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826563 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826538.583146:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626649600/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826563 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826538.583154:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826538.583157:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826538.583160:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626649600:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826538.774207:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826539.158281:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826539.798252:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826540.182218:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826540.822194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826541.206259:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826541.846197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826542.230241:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826542.870208:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826543.254199:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787826543.702280:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826543.702289:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787826543.702293:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826543.702302:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 429
00000100:00080000:2.0:1787826543.702312:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826543.702317:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787826543.702349:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626649728/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826543.702383:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826543.702386:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826543.702388:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826543.702391:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 429
00000100:00080000:2.0:1787826543.702394:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826543.702398:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826543.702407:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826543.702416:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626649856/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826543.702425:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826543.702439:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787826543.702440:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826543.702444:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 429
00000100:00080000:2.0:1787826543.702448:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826543.702452:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826543.702457:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826543.702469:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626649984/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826543.702478:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826543.702481:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826543.702482:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826543.702485:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 429
00000100:00080000:2.0:1787826543.702489:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826543.702492:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826543.702503:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826543.702512:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626650112/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826543.702599:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626649728:192.168.203.145@tcp:250:kworker.0
00000800:00000100:3.0:1787826543.702671:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826543.702709:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826543/real 1787826543]  req@ffff8cd0c3da8000 x1874671626649728/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826568 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826543.702725:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626649728/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826568 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826543.702734:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826543.702738:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826543.702741:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626649728:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826543.702747:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626649856:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826543.702770:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826543.702790:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826543/real 1787826543]  req@ffff8cd0c3dab840 x1874671626649856/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826568 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826543.702802:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626649856/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826568 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826543.702810:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826543.702813:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826543.702816:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626649856:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826543.702822:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626649984:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826543.702841:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826543.702862:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826543/real 1787826543]  req@ffff8cd0c3da8780 x1874671626649984/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826568 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826543.702873:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626649984/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826568 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826543.702880:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826543.702883:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826543.702886:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626649984:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826543.702892:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626650112:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826543.702911:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826543.702921:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826543/real 1787826543]  req@ffff8cd0c3dabc00 x1874671626650112/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826568 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826543.702931:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626650112/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826568 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826543.702938:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826543.702941:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826543.702943:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626650112:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826543.894306:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826544.278208:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826544.918215:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826545.303110:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826545.942147:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826546.326279:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826546.966194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826547.350254:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826547.990186:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826548.374229:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787826548.822216:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826548.822236:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787826548.822239:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826548.822248:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 434
00000100:00080000:2.0:1787826548.822258:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826548.822265:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787826548.822308:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626650240/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826548.822347:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826548.822350:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826548.822351:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826548.822354:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 434
00000100:00080000:2.0:1787826548.822357:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826548.822360:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826548.822368:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826548.822379:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626650368/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826548.822386:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826548.822389:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787826548.822391:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1787826548.822393:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626650240:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787826548.822395:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 434
00000100:00080000:2.0:1787826548.822403:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826548.822406:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826548.822412:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826548.822421:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626650496/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826548.822427:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826548.822429:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826548.822431:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826548.822432:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 434
00000100:00080000:2.0:1787826548.822446:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826548.822450:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826548.822454:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826548.822465:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626650624/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826548.822498:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626650368:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826548.822530:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626650496:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826548.822550:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626650624:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826549.014200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826549.398284:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:2.0:1787826549.910875:0:2343:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:2.0:1787826549.910915:0:2343:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:2.0:1787826549.910928:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787826549.910963:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787826549.910970:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787826549.910976:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:3.0:1787826549.911482:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826548/real 1787826549]  req@ffff8cd0c3daa1c0 x1874671626650368/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826573 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826549.911508:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626650368/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826573 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826549.911520:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826549.911525:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826549.911529:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626650368:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826549.911536:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826548/real 1787826549]  req@ffff8cd0c3da9a40 x1874671626650496/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826573 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826549.911548:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626650496/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826573 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826549.911555:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826549.911559:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826549.911562:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626650496:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826549.911580:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826548/real 1787826549]  req@ffff8cd0c3dab0c0 x1874671626650624/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826573 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826549.911615:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626650624/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826573 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826549.911624:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826549.911628:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826549.911632:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626650624:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826549.911640:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826548/real 1787826549]  req@ffff8cd0c3daa940 x1874671626650240/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826573 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826549.911652:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626650240/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826573 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826549.911659:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826549.911663:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826549.911666:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626650240:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826550.038202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826550.422499:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826551.062197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826551.446389:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826552.086198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826552.470228:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826553.110179:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826553.494316:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787826553.942215:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826553.942224:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787826553.942228:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826553.942236:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 439
00000100:00080000:2.0:1787826553.942247:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826553.942261:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787826553.942301:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab480 x1874671626650752/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826553.942330:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826553.942334:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826553.942335:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826553.942339:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 439
00000100:00080000:2.0:1787826553.942342:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826553.942360:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826553.942370:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826553.942373:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626650752:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787826553.942385:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626650880/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826553.942395:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826553.942399:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787826553.942401:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826553.942404:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 439
00000100:00080000:2.0:1787826553.942415:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826553.942419:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826553.942424:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787826553.942425:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787826553.942438:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626651008/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826553.942447:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826553.942451:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826553.942453:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826553.942457:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 439
00000100:00100000:3.0:1787826553.942460:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826553/real 1787826553]  req@ffff8cd0c3dab480 x1874671626650752/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826578 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787826553.942462:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826553.942470:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826553.942474:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826553.942485:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626651136/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826553.942491:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab480 x1874671626650752/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826578 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826553.942510:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826553.942513:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826553.942516:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626650752:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826553.942533:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626650880:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826553.942565:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826553.942578:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826553/real 1787826553]  req@ffff8cd0c3da83c0 x1874671626650880/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826578 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826553.942589:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626650880/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826578 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826553.942596:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826553.942599:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826553.942602:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626650880:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826553.942608:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626651008:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826553.942626:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826553.942636:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826553/real 1787826553]  req@ffff8cd0c3da9680 x1874671626651008/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826578 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826553.942648:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626651008/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826578 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826553.942663:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826553.942666:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826553.942668:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626651008:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826553.942671:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626651136:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826553.942686:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826553.942692:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826553/real 1787826553]  req@ffff8cd0c3da8f00 x1874671626651136/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826578 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826553.942699:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626651136/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826578 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826553.942703:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826553.942705:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826553.942706:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626651136:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826554.134182:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826554.518214:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826555.158194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826555.542234:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826556.182275:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826556.566207:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826557.206196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826557.590254:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826558.230208:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826558.614222:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787826559.062199:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826559.062209:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787826559.062226:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826559.062235:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 444
00000100:00080000:1.0:1787826559.062246:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826559.062252:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787826559.062282:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626651264/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826559.062306:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826559.062309:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826559.062311:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826559.062315:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 444
00000100:00080000:1.0:1787826559.062319:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826559.062323:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826559.062332:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826559.062337:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626651264:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787826559.062347:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626651392/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826559.062358:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826559.062361:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787826559.062363:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826559.062367:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 444
00000100:00080000:1.0:1787826559.062371:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826559.062375:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826559.062379:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826559.062390:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626651520/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787826559.062396:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787826559.062415:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826559.062418:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826559.062420:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826559.062424:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 444
00000100:00080000:1.0:1787826559.062428:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826559.062433:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826559.062438:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826559.062440:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826559/real 1787826559]  req@ffff8cd0d48f61c0 x1874671626651264/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826584 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787826559.062452:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626651648/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826559.062515:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626651264/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826584 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826559.062526:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826559.062530:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826559.062534:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626651264:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826559.062556:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626651392:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826559.062580:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826559.062593:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826559/real 1787826559]  req@ffff8cd0d48f7c00 x1874671626651392/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826584 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826559.062605:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626651392/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826584 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826559.062613:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826559.062616:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826559.062619:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626651392:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826559.062625:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626651520:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826559.062646:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826559.062656:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826559/real 1787826559]  req@ffff8cd0d48f52c0 x1874671626651520/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826584 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826559.062667:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626651520/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826584 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826559.062687:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826559.062691:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826559.062694:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626651520:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826559.062699:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626651648:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826559.062720:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826559.062730:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826559/real 1787826559]  req@ffff8cd0d48f6d00 x1874671626651648/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826584 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826559.062740:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626651648/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826584 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826559.062748:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826559.062751:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826559.062753:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626651648:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826559.254194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826559.638256:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826560.278200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826560.662226:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826561.302150:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826561.686257:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826562.326216:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826562.710213:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826563.350250:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826563.734406:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787826564.182274:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826564.182300:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787826564.182304:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826564.182314:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 450
00000100:00080000:2.0:1787826564.182324:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826564.182331:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787826564.182365:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8b40 x1874671626651776/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826564.182399:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826564.182403:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826564.182405:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826564.182408:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 450
00000100:00080000:2.0:1787826564.182412:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826564.182415:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826564.182424:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826564.182437:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626651904/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826564.182446:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826564.182449:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787826564.182451:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826564.182454:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 450
00000100:00080000:2.0:1787826564.182457:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826564.182461:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826564.182477:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826564.182488:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626652032/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826564.182501:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826564.182503:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826564.182505:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826564.182507:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 450
00000100:00080000:2.0:1787826564.182510:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826564.182512:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826564.182516:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826564.182523:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626652160/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826564.182603:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626651776:192.168.203.145@tcp:250:kworker.0
00000800:00000100:3.0:1787826564.182658:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826564.182695:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826564/real 1787826564]  req@ffff8cd0c3da8b40 x1874671626651776/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826589 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826564.182717:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8b40 x1874671626651776/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826589 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826564.182727:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826564.182732:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826564.182735:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626651776:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826564.182741:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626651904:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826564.182764:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826564.182776:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826564/real 1787826564]  req@ffff8cd0c3da8f00 x1874671626651904/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826589 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826564.182787:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626651904/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826589 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826564.182794:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826564.182797:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826564.182799:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626651904:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826564.182805:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626652032:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826564.182824:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826564.182834:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826564/real 1787826564]  req@ffff8cd0c3da9680 x1874671626652032/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826589 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826564.182844:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626652032/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826589 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826564.182865:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826564.182868:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826564.182871:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626652032:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826564.182876:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626652160:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826564.182897:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826564.182908:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826564/real 1787826564]  req@ffff8cd0c3da83c0 x1874671626652160/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826589 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826564.182917:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626652160/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826589 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826564.182924:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826564.182926:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826564.182929:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626652160:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826564.374308:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826564.758230:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826565.398237:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826565.782281:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826566.422318:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826566.806310:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826567.446195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826567.830277:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826568.470225:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826568.854189:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787826569.302200:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826569.302210:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787826569.302214:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826569.302222:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 455
00000100:00080000:2.0:1787826569.302232:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826569.302239:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787826569.302271:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab480 x1874671626652288/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826569.302301:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826569.302305:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826569.302307:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826569.302310:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 455
00000100:00080000:2.0:1787826569.302314:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826569.302317:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826569.302338:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826569.302351:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626652416/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826569.302357:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626652288:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787826569.302367:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826569.302371:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787826569.302373:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826569.302378:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 455
00000100:00080000:2.0:1787826569.302382:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826569.302386:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826569.302391:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826569.302401:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626652544/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826569.302410:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826569.302413:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826569.302415:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826569.302421:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 455
00000800:00000100:3.0:1787826569.302421:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787826569.302428:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826569.302443:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826569.302448:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826569.302462:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626652672/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826569.302475:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826569/real 1787826569]  req@ffff8cd0c3dab480 x1874671626652288/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826594 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826569.302518:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab480 x1874671626652288/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826594 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826569.302526:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826569.302530:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826569.302533:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626652288:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826569.302552:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626652416:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826569.302574:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826569.302585:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826569/real 1787826569]  req@ffff8cd0c3daa940 x1874671626652416/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826594 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826569.302595:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626652416/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826594 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826569.302601:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826569.302604:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826569.302607:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626652416:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826569.302611:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626652544:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826569.302626:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826569.302634:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826569/real 1787826569]  req@ffff8cd0c3dab0c0 x1874671626652544/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826594 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826569.302644:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626652544/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826594 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826569.302660:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826569.302663:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826569.302666:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626652544:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826569.302670:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626652672:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826569.302687:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826569.302706:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826569/real 1787826569]  req@ffff8cd0c3da9a40 x1874671626652672/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826594 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826569.302717:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626652672/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826594 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826569.302723:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826569.302725:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826569.302728:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626652672:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826569.494231:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826569.878675:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826570.518194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826570.902186:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826571.542234:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826571.926373:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826572.566182:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826572.951330:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826573.590216:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826573.974290:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787826574.422252:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826574.422262:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787826574.422283:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826574.422293:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 460
00000100:00080000:2.0:1787826574.422303:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826574.422310:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787826574.422357:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626652800/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826574.422407:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826574.422411:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826574.422413:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826574.422417:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 460
00000100:00080000:2.0:1787826574.422421:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826574.422425:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826574.422435:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826574.422447:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626652928/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826574.422456:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826574.422460:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787826574.422461:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826574.422465:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 460
00000100:00080000:2.0:1787826574.422468:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826574.422471:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826574.422477:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826574.422499:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626653056/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826574.422520:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826574.422523:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826574.422525:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826574.422529:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 460
00000100:00080000:2.0:1787826574.422533:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826574.422548:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826574.422554:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826574.422566:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626653184/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826574.422613:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626652800:192.168.203.145@tcp:250:kworker.0
00000800:00000100:3.0:1787826574.422691:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826574.422733:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826574/real 1787826574]  req@ffff8cd0c3daa1c0 x1874671626652800/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826599 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826574.422757:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626652800/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826599 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826574.422768:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826574.422772:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826574.422776:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626652800:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826574.422783:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626652928:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826574.422808:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826574.422820:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826574/real 1787826574]  req@ffff8cd0c3dabc00 x1874671626652928/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826599 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826574.422831:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626652928/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826599 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826574.422839:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826574.422842:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826574.422845:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626652928:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826574.422850:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626653056:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826574.422869:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826574.422880:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826574/real 1787826574]  req@ffff8cd0c3da8780 x1874671626653056/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826599 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826574.422891:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626653056/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826599 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826574.422913:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826574.422917:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826574.422920:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626653056:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826574.422925:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626653184:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826574.422948:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826574.422960:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826574/real 1787826574]  req@ffff8cd0c3dab840 x1874671626653184/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826599 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826574.422971:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626653184/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826599 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826574.422978:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826574.422982:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826574.422985:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626653184:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826574.614167:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826574.998273:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826575.638229:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826576.022666:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826576.662204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826577.046311:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826577.686202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826578.070278:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826578.710189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826579.094279:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787826579.542278:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826579.542291:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787826579.542296:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826579.542308:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 465
00000100:00080000:1.0:1787826579.542322:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826579.542330:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787826579.542389:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626653312/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826579.542423:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826579.542426:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826579.542428:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826579.542432:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 465
00000100:00080000:1.0:1787826579.542436:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826579.542440:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826579.542479:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826579.542496:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626653440/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826579.542511:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787826579.542514:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626653312:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787826579.542516:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787826579.542519:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826579.542530:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 465
00000100:00080000:1.0:1787826579.542535:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826579.542540:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826579.542545:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826579.542560:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626653568/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826579.542572:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826579.542584:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826579.542587:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826579.542591:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 465
00000100:00080000:1.0:1787826579.542596:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1787826579.542599:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787826579.542604:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826579.542610:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826579.542633:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626653696/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826579.542650:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826579/real 1787826579]  req@ffff8cd0d48f6d00 x1874671626653312/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826604 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826579.542696:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626653312/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826604 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826579.542707:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826579.542712:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826579.542716:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626653312:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826579.542740:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626653440:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826579.542765:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826579.542779:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826579/real 1787826579]  req@ffff8cd0d48f52c0 x1874671626653440/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826604 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826579.542791:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626653440/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826604 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826579.542799:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826579.542802:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826579.542806:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626653440:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826579.542812:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626653568:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826579.542831:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826579.542843:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826579/real 1787826579]  req@ffff8cd0d48f7c00 x1874671626653568/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826604 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826579.542855:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626653568/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826604 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826579.542873:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826579.542877:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826579.542880:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626653568:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826579.542886:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626653696:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826579.542907:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826579.542918:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826579/real 1787826579]  req@ffff8cd0d48f61c0 x1874671626653696/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826604 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826579.542930:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626653696/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826604 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826579.542937:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826579.542940:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826579.542943:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626653696:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826579.734201:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826580.118526:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826580.758216:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826581.142278:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826581.782263:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826582.166308:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826582.806211:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826583.190291:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826583.830200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826584.214261:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787826584.662213:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826584.662227:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787826584.662255:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826584.662268:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 470
00000100:00080000:1.0:1787826584.662283:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826584.662292:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787826584.662353:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626653824/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826584.662388:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826584.662392:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826584.662394:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826584.662398:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 470
00000100:00080000:1.0:1787826584.662401:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826584.662406:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826584.662419:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826584.662432:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626653952/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826584.662442:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826584.662446:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787826584.662448:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826584.662453:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 470
00000100:00080000:1.0:1787826584.662456:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826584.662463:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787826584.662470:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626653824:192.168.203.145@tcp:250:kworker.0
00000008:00100000:1.0:1787826584.662472:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826584.662492:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626654080/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826584.662520:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826584.662524:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826584.662526:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826584.662531:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 470
00000100:00080000:1.0:1787826584.662538:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826584.662543:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826584.662549:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826584.662566:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626654208/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826584.662606:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626653952:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826584.662676:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626654080:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826584.662704:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626654208:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826584.854283:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826585.238473:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826585.878203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826586.263337:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826586.902257:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826587.286249:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:0.0:1787826587.734428:0:2345:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:0.0:1787826587.734470:0:2345:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:0.0:1787826587.734486:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:0.0:1787826587.734543:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:0.0:1787826587.734552:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:0.0:1787826587.734559:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:3.0:1787826587.734580:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826584/real 1787826587]  req@ffff8cd0d48f70c0 x1874671626654080/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826609 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826587.734606:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626654080/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826609 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826587.734618:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826587.734622:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826587.734626:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626654080:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826587.734634:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826584/real 1787826587]  req@ffff8cd0d48f43c0 x1874671626654208/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826609 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826587.734658:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626654208/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826609 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826587.734665:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826587.734669:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826587.734673:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626654208:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826587.734679:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826584/real 1787826587]  req@ffff8cd0d48f5a40 x1874671626653824/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826609 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826587.734718:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626653824/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826609 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826587.734726:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826587.734730:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826587.734733:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626653824:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826587.734739:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826584/real 1787826587]  req@ffff8cd0d48f7840 x1874671626653952/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826609 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826587.734751:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626653952/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826609 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826587.734759:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826587.734762:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826587.734765:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626653952:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826587.926187:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826588.310217:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826588.950211:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826589.334339:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787826589.782419:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826589.782430:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787826589.782433:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826589.782443:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 475
00000100:00080000:0.0:1787826589.782454:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826589.782462:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826589.782496:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d2c0 x1874671626654336/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826589.782526:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826589.782530:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826589.782532:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826589.782536:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 475
00000100:00080000:0.0:1787826589.782540:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826589.782543:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826589.782566:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826589.782581:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626654464/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826589.782592:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826589.782595:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787826589.782597:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826589.782614:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 475
00000100:00080000:0.0:1787826589.782619:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826589.782623:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826589.782628:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826589.782638:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626654592/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826589.782647:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826589.782661:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826589.782663:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826589.782667:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 475
00000100:00080000:0.0:1787826589.782671:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826589.782675:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826589.782680:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826589.782690:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626654720/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826589.782877:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626654336:192.168.203.145@tcp:250:kworker.0
00000800:00000100:3.0:1787826589.782947:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826589.782995:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826589/real 1787826589]  req@ffff8cd0c8e9d2c0 x1874671626654336/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826614 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826589.783017:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d2c0 x1874671626654336/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826614 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826589.783038:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826589.784166:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826589.784178:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626654336:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826589.784189:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626654464:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826589.784249:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826589.784273:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826589/real 1787826589]  req@ffff8cd0c8e9c3c0 x1874671626654464/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826614 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826589.784286:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626654464/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826614 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826589.784305:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826589.784308:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826589.784312:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626654464:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826589.784318:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626654592:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826589.784341:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826589.784352:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826589/real 1787826589]  req@ffff8cd0c8e9cb40 x1874671626654592/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826614 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826589.784362:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626654592/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826614 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826589.784368:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826589.784372:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826589.784374:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626654592:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826589.784394:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626654720:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826589.784418:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826589.784429:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826589/real 1787826589]  req@ffff8cd0c8e9f480 x1874671626654720/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826614 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826589.784441:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626654720/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826614 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826589.784448:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826589.784451:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826589.784453:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626654720:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826589.974206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826590.358315:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826590.998527:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826591.382231:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826592.022226:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826592.406281:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826593.046302:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826593.430292:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826594.070281:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826594.454299:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787826594.902212:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826594.902223:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787826594.902227:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826594.902236:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 480
00000100:00080000:1.0:1787826594.902246:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826594.902263:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787826594.902298:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626654848/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826594.902324:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826594.902327:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826594.902328:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826594.902330:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 480
00000100:00080000:1.0:1787826594.902333:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826594.902335:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826594.902354:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826594.902364:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626654976/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826594.902372:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826594.902375:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:3.0:1787826594.902376:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626654848:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787826594.902377:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826594.902383:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 480
00000100:00080000:1.0:1787826594.902387:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826594.902390:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826594.902392:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826594.902399:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626655104/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826594.902408:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826594.902411:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826594.902413:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826594.902417:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 480
00000100:00080000:1.0:1787826594.902420:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826594.902424:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826594.902429:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787826594.902437:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787826594.902442:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626655232/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826594.902476:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826594/real 1787826594]  req@ffff8cd0d48f4000 x1874671626654848/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826619 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826594.902493:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626654848/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826619 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826594.902503:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826594.902513:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826594.902517:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626654848:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826594.902537:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626654976:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826594.902565:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826594.902576:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826594/real 1787826594]  req@ffff8cd0d48f6580 x1874671626654976/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826619 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826594.902587:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626654976/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826619 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826594.902595:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826594.902597:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826594.902601:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626654976:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826594.902605:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626655104:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826594.902618:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826594.902624:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826594/real 1787826594]  req@ffff8cd0d48f7480 x1874671626655104/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826619 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826594.902631:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626655104/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826619 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826594.902636:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826594.902637:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826594.902639:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626655104:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826594.902642:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626655232:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826594.902668:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826594.902675:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826594/real 1787826594]  req@ffff8cd0d48f4b40 x1874671626655232/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826619 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826594.902681:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626655232/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826619 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826594.902686:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826594.902688:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826594.902689:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626655232:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826595.094203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826595.478235:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826596.118182:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826596.502264:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826597.142176:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826597.526207:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826598.166202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826598.550263:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826599.190201:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826599.574191:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787826600.022209:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826600.022227:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787826600.022231:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826600.022240:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 485
00000100:00080000:0.0:1787826600.022251:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826600.022257:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826600.022292:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626655360/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826600.022317:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826600.022321:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826600.022322:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826600.022326:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 485
00000100:00080000:0.0:1787826600.022329:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826600.022333:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826600.022341:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826600.022353:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626655488/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826600.022362:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826600.022366:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:3.0:1787826600.022369:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626655360:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787826600.022370:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826600.022377:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 485
00000100:00080000:0.0:1787826600.022381:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826600.022397:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826600.022402:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826600.022416:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626655616/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826600.022425:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826600.022428:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826600.022430:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826600.022434:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 485
00000100:00080000:0.0:1787826600.022439:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1787826600.022442:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787826600.022446:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826600.022452:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826600.022464:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626655744/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826600.022490:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826600/real 1787826600]  req@ffff8cd0c8e9c000 x1874671626655360/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826625 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826600.022507:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626655360/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826625 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826600.022517:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826600.022521:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826600.022525:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626655360:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826600.022557:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626655488:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826600.022580:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826600.022591:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826600/real 1787826600]  req@ffff8cd0c8e9f840 x1874671626655488/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826625 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826600.022603:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626655488/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826625 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826600.022611:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826600.022614:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826600.022618:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626655488:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826600.022623:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626655616:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826600.022640:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826600.022650:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826600/real 1787826600]  req@ffff8cd0c8e9ed00 x1874671626655616/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826625 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826600.022660:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626655616/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826625 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826600.022668:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826600.022671:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826600.022674:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626655616:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826600.022679:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626655744:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826600.022695:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826600.022705:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826600/real 1787826600]  req@ffff8cd0c8e9d680 x1874671626655744/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826625 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826600.022725:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626655744/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826625 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826600.022731:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826600.022734:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826600.022737:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626655744:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826600.214159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826600.598270:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826601.238271:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826601.622239:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826602.262195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826602.646353:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826603.286206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826603.670266:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826604.310269:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826604.694315:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787826605.142260:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826605.142273:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787826605.142300:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826605.142310:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 490
00000100:00080000:1.0:1787826605.142322:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826605.142328:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787826605.142367:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626655872/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826605.142394:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826605.142396:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826605.142398:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826605.142400:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 490
00000100:00080000:1.0:1787826605.142402:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826605.142405:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826605.142414:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826605.142425:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626656000/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826605.142436:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826605.142440:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787826605.142442:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826605.142446:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 490
00000100:00080000:1.0:1787826605.142449:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826605.142453:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826605.142459:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826605.142461:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626655872:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787826605.142472:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626656128/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826605.142497:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826605.142500:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826605.142501:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826605.142505:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 490
00000100:00080000:1.0:1787826605.142509:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826605.142512:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826605.142517:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826605.142530:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626656256/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787826605.142543:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826605.142594:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826605/real 1787826605]  req@ffff8cd0d48f4b40 x1874671626655872/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826630 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826605.142612:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626655872/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826630 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826605.142621:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826605.142625:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826605.142629:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626655872:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826605.142636:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626656000:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826605.142666:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826605.142678:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826605/real 1787826605]  req@ffff8cd0d48f7480 x1874671626656000/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826630 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826605.142688:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626656000/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826630 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826605.142695:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826605.142698:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826605.142701:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626656000:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826605.142727:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626656128:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826605.142744:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826605.142752:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826605/real 1787826605]  req@ffff8cd0d48f6580 x1874671626656128/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826630 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826605.142762:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626656128/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826630 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826605.142768:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826605.142771:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826605.142774:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626656128:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826605.142778:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626656256:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826605.142792:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826605.142800:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826605/real 1787826605]  req@ffff8cd0d48f4000 x1874671626656256/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826630 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826605.142836:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626656256/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826630 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826605.142844:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826605.142847:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826605.142850:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626656256:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826605.334242:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826605.718193:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826606.358200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826606.742389:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826607.382239:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826607.766589:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826608.406237:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826608.790319:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826609.430183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826609.814234:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787826610.262162:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826610.262172:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787826610.262176:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826610.262185:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 496
00000100:00080000:1.0:1787826610.262198:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826610.262204:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787826610.262235:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626656384/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826610.262253:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826610.262256:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826610.262258:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826610.262261:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 496
00000100:00080000:1.0:1787826610.262264:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826610.262268:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826610.262290:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826610.262297:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626656384:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787826610.262305:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626656512/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826610.262313:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826610.262316:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787826610.262317:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826610.262320:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 496
00000100:00080000:1.0:1787826610.262323:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826610.262326:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826610.262330:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826610.262338:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626656640/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826610.262347:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826610.262350:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826610.262351:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826610.262356:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 496
00000800:00000100:3.0:1787826610.262361:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787826610.262362:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826610.262368:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826610.262374:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826610.262387:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626656768/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826610.262411:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826610/real 1787826610]  req@ffff8cd0d48f7840 x1874671626656384/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826635 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826610.262424:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626656384/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826635 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826610.262431:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826610.262433:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826610.262441:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626656384:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826610.262458:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626656512:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826610.262486:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826610.262499:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826610/real 1787826610]  req@ffff8cd0d48f5a40 x1874671626656512/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826635 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826610.262511:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626656512/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826635 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826610.262519:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826610.262522:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826610.262526:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626656512:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826610.262531:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626656640:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826610.262559:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826610.262570:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826610/real 1787826610]  req@ffff8cd0d48f43c0 x1874671626656640/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826635 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826610.262581:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626656640/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826635 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826610.262599:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826610.262602:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826610.262605:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626656640:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826610.262611:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626656768:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826610.262631:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826610.262642:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826610/real 1787826610]  req@ffff8cd0d48f70c0 x1874671626656768/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826635 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826610.262667:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626656768/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826635 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826610.262673:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826610.262675:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826610.262677:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626656768:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826610.454211:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826610.838288:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826611.478182:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826611.862255:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826612.502223:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826612.886306:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826613.526247:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826613.910309:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826614.550217:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826614.934286:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787826615.382230:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826615.382236:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787826615.382239:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826615.382247:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 501
00000100:00080000:0.0:1787826615.382256:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826615.382275:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826615.382303:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626656896/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826615.382323:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826615.382325:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826615.382331:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826615.382333:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 501
00000100:00080000:0.0:1787826615.382336:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826615.382338:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826615.382345:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826615.382353:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626657024/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826615.382359:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826615.382361:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787826615.382363:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826615.382365:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 501
00000100:00080000:0.0:1787826615.382367:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826615.382368:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826615.382371:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826615.382376:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626656896:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826615.382379:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626657152/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826615.382388:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826615.382389:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826615.382395:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826615.382398:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 501
00000100:00080000:0.0:1787826615.382401:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826615.382403:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826615.382407:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826615.382414:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626657280/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787826615.382457:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826615.382555:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826615/real 1787826615]  req@ffff8cd0c8e9de00 x1874671626656896/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826640 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826615.382573:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626656896/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826640 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826615.382582:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826615.382586:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826615.382590:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626656896:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826615.382598:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626657024:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826615.382641:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826615.382655:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826615/real 1787826615]  req@ffff8cd0c8e9e940 x1874671626657024/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826640 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826615.382667:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626657024/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826640 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826615.382675:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826615.382678:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826615.382681:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626657024:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826615.382704:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626657152:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826615.382723:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826615.382733:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826615/real 1787826615]  req@ffff8cd0c8e9e580 x1874671626657152/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826640 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826615.382755:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626657152/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826640 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826615.382763:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826615.382766:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826615.382769:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626657152:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826615.382775:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626657280:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826615.382792:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826615.382802:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826615/real 1787826615]  req@ffff8cd0c8e9cf00 x1874671626657280/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826640 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826615.382828:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626657280/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826640 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826615.382848:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826615.382851:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826615.382854:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626657280:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826615.574218:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826615.958275:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826616.598242:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826616.982302:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826617.622154:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826618.006284:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826618.646687:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826619.030261:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826619.670198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826620.054403:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787826620.502208:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826620.502215:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787826620.502218:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826620.502227:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 506
00000100:00080000:0.0:1787826620.502236:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826620.502242:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826620.502282:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626657408/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826620.502301:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826620.502304:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826620.502305:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826620.502307:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 506
00000100:00080000:0.0:1787826620.502309:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826620.502311:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826620.502320:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826620.502335:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626657536/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826620.502345:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626657408:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787826620.502367:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826620.502371:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787826620.502373:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826620.502378:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 506
00000100:00080000:0.0:1787826620.502383:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826620.502396:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:3.0:1787826620.502402:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:0.0:1787826620.502403:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826620.502418:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626657664/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826620.502428:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826620.502432:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826620.502434:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826620.502438:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 506
00000100:00080000:0.0:1787826620.502441:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826620.502443:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826620.502447:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826620.502458:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626657792/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826620.502460:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826620/real 1787826620]  req@ffff8cd0c8e9fc00 x1874671626657408/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826645 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826620.502488:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626657408/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826645 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826620.502498:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826620.502502:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826620.502506:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626657408:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826620.502526:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626657536:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826620.502554:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826620.502567:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826620/real 1787826620]  req@ffff8cd0c8e9cf00 x1874671626657536/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826645 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826620.502578:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626657536/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826645 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826620.502590:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826620.502593:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826620.502596:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626657536:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826620.502602:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626657664:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826620.502623:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826620.502631:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826620/real 1787826620]  req@ffff8cd0c8e9e580 x1874671626657664/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826645 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826620.502638:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626657664/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826645 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826620.502643:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826620.502644:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826620.502646:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626657664:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826620.502649:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626657792:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826620.502659:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826620.502664:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826620/real 1787826620]  req@ffff8cd0c8e9e940 x1874671626657792/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826645 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826620.502680:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626657792/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826645 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826620.502687:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826620.502690:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826620.502693:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626657792:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826620.694199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826621.078250:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826621.718237:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826622.102832:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826622.742253:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826623.126283:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826623.766229:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826624.150314:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826624.790192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826625.174243:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787826625.622284:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826625.622292:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787826625.622295:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826625.622303:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 511
00000100:00080000:1.0:1787826625.622311:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826625.622316:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787826625.622347:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626657920/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826625.622380:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826625.622383:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826625.622385:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826625.622388:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 511
00000100:00080000:1.0:1787826625.622392:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826625.622395:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826625.622406:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826625.622419:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626658048/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826625.622447:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826625.622450:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787826625.622452:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826625.622456:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 511
00000100:00080000:1.0:1787826625.622459:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1787826625.622476:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626657920:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787826625.622477:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826625.622487:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826625.622502:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626658176/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826625.622511:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826625.622514:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826625.622516:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826625.622520:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 511
00000100:00080000:1.0:1787826625.622525:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826625.622529:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826625.622533:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787826625.622541:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787826625.622546:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626658304/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826625.622584:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826625/real 1787826625]  req@ffff8cd0d48f61c0 x1874671626657920/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826650 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826625.622603:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626657920/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826650 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826625.622609:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826625.622612:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826625.622620:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626657920:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826625.622637:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626658048:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826625.622659:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826625.622667:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826625/real 1787826625]  req@ffff8cd0d48f7c00 x1874671626658048/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826650 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826625.622674:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626658048/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826650 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826625.622679:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826625.622681:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826625.622683:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626658048:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826625.622687:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626658176:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826625.622697:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826625.622702:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826625/real 1787826625]  req@ffff8cd0d48f52c0 x1874671626658176/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826650 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826625.622709:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626658176/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826650 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826625.622713:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826625.622715:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826625.622717:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626658176:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826625.622720:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626658304:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826625.622729:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826625.622734:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826625/real 1787826625]  req@ffff8cd0d48f6d00 x1874671626658304/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826650 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826625.622749:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626658304/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826650 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826625.622754:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826625.622756:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826625.622758:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626658304:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826625.814201:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826626.198329:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826626.838199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826627.222476:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826627.862220:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826628.246265:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826628.886281:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826629.270474:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826629.910204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826630.294397:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787826630.742231:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826630.742243:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787826630.742249:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826630.742261:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 516
00000100:00080000:1.0:1787826630.742274:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826630.742283:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787826630.742332:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626658432/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826630.742385:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826630.742389:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826630.742391:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826630.742395:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 516
00000100:00080000:1.0:1787826630.742399:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826630.742403:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826630.742416:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826630.742430:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626658560/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826630.742439:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826630.742444:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787826630.742446:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826630.742449:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 516
00000100:00080000:1.0:1787826630.742453:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826630.742458:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826630.742463:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826630.742475:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626658688/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826630.742486:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826630.742489:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826630.742490:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826630.742494:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 516
00000100:00080000:1.0:1787826630.742502:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826630.742506:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826630.742511:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826630.742520:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626658816/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826630.742538:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626658432:192.168.203.145@tcp:250:kworker.0
00000800:00000100:3.0:1787826630.742613:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826630.742656:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826630/real 1787826630]  req@ffff8cd0d48f6d00 x1874671626658432/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826655 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826630.742671:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626658432/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826655 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826630.742681:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826630.742685:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826630.742689:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626658432:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826630.742697:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626658560:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826630.742735:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826630.742748:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826630/real 1787826630]  req@ffff8cd0d48f52c0 x1874671626658560/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826655 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826630.742760:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626658560/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826655 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826630.742768:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826630.742771:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826630.742774:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626658560:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826630.742808:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626658688:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826630.742827:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826630.742837:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826630/real 1787826630]  req@ffff8cd0d48f7c00 x1874671626658688/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826655 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826630.742848:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626658688/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826655 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826630.742855:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826630.742858:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826630.742861:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626658688:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826630.742867:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626658816:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826630.742883:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826630.742892:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826630/real 1787826630]  req@ffff8cd0d48f61c0 x1874671626658816/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826655 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826630.742914:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626658816/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826655 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826630.742922:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826630.742924:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826630.742927:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626658816:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826630.934204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826631.318532:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826631.958206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826632.342423:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826632.982354:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826633.366274:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826634.006203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826634.390697:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826635.030199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826635.414827:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787826635.862437:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826635.862447:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787826635.862452:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826635.862461:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 521
00000100:00080000:0.0:1787826635.862486:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826635.862492:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826635.862521:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626658944/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826635.862548:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826635.862551:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826635.862553:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826635.862556:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 521
00000100:00080000:0.0:1787826635.862559:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826635.862562:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826635.862571:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826635.862580:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626659072/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826635.862590:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826635.862592:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787826635.862594:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826635.862597:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 521
00000100:00080000:0.0:1787826635.862600:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826635.862603:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826635.862607:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826635.862614:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626659200/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826635.862622:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826635.862630:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826635.862631:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826635.862635:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 521
00000100:00080000:0.0:1787826635.862638:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826635.862642:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826635.862646:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826635.862654:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626659328/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826635.863015:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626658944:192.168.203.145@tcp:250:kworker.0
00000800:00000100:3.0:1787826635.863071:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826635.863138:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826635/real 1787826635]  req@ffff8cd0c8e9de00 x1874671626658944/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826660 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826635.863152:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626658944/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826660 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826635.863161:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826635.863164:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826635.863167:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626658944:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826635.863174:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626659072:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826635.863201:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826635.863211:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826635/real 1787826635]  req@ffff8cd0c8e9d680 x1874671626659072/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826660 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826635.863221:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626659072/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826660 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826635.863227:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826635.863230:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826635.863232:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626659072:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826635.863237:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626659200:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826635.863252:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826635.863260:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826635/real 1787826635]  req@ffff8cd0c8e9ed00 x1874671626659200/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826660 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826635.863268:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626659200/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826660 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826635.863275:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826635.863278:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826635.863280:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626659200:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826635.863285:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626659328:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826635.863300:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826635.863309:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826635/real 1787826635]  req@ffff8cd0c8e9f840 x1874671626659328/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826660 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826635.863330:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626659328/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826660 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826635.863337:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826635.863340:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826635.863343:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626659328:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826636.054600:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826636.438283:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826637.078436:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826637.462876:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826638.102197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826638.486281:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826639.126291:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826639.510363:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826640.150195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826640.534205:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787826640.982223:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826640.982233:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787826640.982237:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826640.982253:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 526
00000100:00080000:0.0:1787826640.982264:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826640.982271:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826640.982311:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626659456/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826640.982344:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826640.982347:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826640.982349:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826640.982352:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 526
00000100:00080000:0.0:1787826640.982356:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826640.982360:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826640.982369:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826640.982382:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626659584/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826640.982408:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826640.982411:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787826640.982414:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826640.982418:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 526
00000100:00080000:0.0:1787826640.982422:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826640.982425:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826640.982431:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826640.982444:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626659712/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826640.982454:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626659456:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787826640.982460:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826640.982469:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826640.982471:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826640.982476:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 526
00000100:00080000:0.0:1787826640.982479:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826640.982483:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826640.982488:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826640.982502:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626659840/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787826640.984227:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826640.984291:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826640/real 1787826640]  req@ffff8cd0c8e9c000 x1874671626659456/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826665 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826640.984316:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626659456/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826665 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826640.984330:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826640.984335:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826640.984339:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626659456:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826640.984350:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626659584:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826640.984400:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826640.984413:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826640/real 1787826640]  req@ffff8cd0c8e9f480 x1874671626659584/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826665 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826640.984426:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626659584/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826665 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826640.984434:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826640.984438:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826640.984441:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626659584:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826640.984474:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626659712:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826640.984497:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826640.984509:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826640/real 1787826640]  req@ffff8cd0c8e9cb40 x1874671626659712/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826665 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826640.984521:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626659712/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826665 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826640.984530:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826640.984533:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826640.984536:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626659712:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826640.984542:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626659840:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826640.984562:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826640.984573:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826640/real 1787826640]  req@ffff8cd0c8e9c3c0 x1874671626659840/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826665 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826640.984598:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626659840/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826665 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826640.984606:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826640.984609:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826640.984613:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626659840:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826641.174215:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826641.558591:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826642.198196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826642.582785:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826643.222223:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826643.606280:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826644.247330:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826644.630353:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826645.270205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826645.654534:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787826646.102277:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826646.102284:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787826646.102288:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826646.102301:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 531
00000100:00080000:1.0:1787826646.102309:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826646.102315:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787826646.102343:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626659968/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826646.102372:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826646.102379:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826646.102381:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826646.102384:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 531
00000100:00080000:1.0:1787826646.102388:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826646.102392:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826646.102410:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826646.102422:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626660096/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826646.102432:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826646.102450:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787826646.102452:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826646.102456:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 531
00000100:00080000:1.0:1787826646.102459:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826646.102463:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826646.102468:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826646.102480:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626660224/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826646.102489:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826646.102492:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826646.102493:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826646.102497:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 531
00000100:00080000:1.0:1787826646.102500:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826646.102514:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826646.102519:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826646.102533:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626660352/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826646.102945:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626659968:192.168.203.145@tcp:250:kworker.0
00000800:00000100:3.0:1787826646.103007:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826646.103042:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826646/real 1787826646]  req@ffff8cd0d48f70c0 x1874671626659968/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826671 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826646.103056:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626659968/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826671 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826646.103063:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826646.103065:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826646.103069:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626659968:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826646.103074:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626660096:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826646.103130:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826646.103143:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826646/real 1787826646]  req@ffff8cd0d48f43c0 x1874671626660096/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826671 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826646.103155:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626660096/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826671 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826646.103161:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826646.103163:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826646.103165:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626660096:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826646.103171:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626660224:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826646.103192:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826646.103204:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826646/real 1787826646]  req@ffff8cd0d48f5a40 x1874671626660224/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826671 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826646.103215:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626660224/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826671 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826646.103223:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826646.103226:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826646.103230:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626660224:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826646.103237:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626660352:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826646.103255:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826646.103266:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826646/real 1787826646]  req@ffff8cd0d48f7840 x1874671626660352/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826671 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826646.103296:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626660352/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826671 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826646.103301:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826646.103303:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826646.103306:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626660352:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826646.294200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826646.678704:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826647.318195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826647.702271:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826648.342247:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826648.726737:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826649.366207:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826649.750545:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826650.390215:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826650.775251:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787826651.222234:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826651.222244:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787826651.222249:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826651.222258:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 537
00000100:00080000:0.0:1787826651.222285:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826651.222292:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826651.222332:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d2c0 x1874671626660480/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826651.222365:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826651.222368:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826651.222370:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826651.222374:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 537
00000100:00080000:0.0:1787826651.222377:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826651.222381:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826651.222390:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826651.222414:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626660608/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826651.222424:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826651.222427:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787826651.222429:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826651.222432:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 537
00000100:00080000:0.0:1787826651.222436:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826651.222440:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826651.222444:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826651.222454:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626660736/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826651.222462:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826651.222470:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826651.222472:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826651.222477:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 537
00000100:00080000:0.0:1787826651.222481:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826651.222484:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826651.222489:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826651.222500:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626660864/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826651.222588:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626660480:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826651.222671:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626660608:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826651.222704:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626660736:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826651.222724:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626660864:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826651.414219:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826651.798277:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826652.438228:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826652.822240:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826653.462230:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826653.846267:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:0.0:1787826654.294322:0:2344:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:0.0:1787826654.294366:0:2344:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:0.0:1787826654.294382:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:0.0:1787826654.294418:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:0.0:1787826654.294427:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:0.0:1787826654.294435:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:3.0:1787826654.294463:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826651/real 1787826654]  req@ffff8cd0c8e9d2c0 x1874671626660480/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826676 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826654.294514:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d2c0 x1874671626660480/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826676 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826654.294525:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826654.294530:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826654.294534:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626660480:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826654.294542:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826651/real 1787826654]  req@ffff8cd0c8e9c3c0 x1874671626660608/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826676 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826654.294554:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626660608/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826676 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826654.294561:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826654.294565:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826654.294568:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626660608:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826654.294574:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826651/real 1787826654]  req@ffff8cd0c8e9cb40 x1874671626660736/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826676 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826654.294586:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626660736/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826676 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826654.294593:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826654.294596:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826654.294599:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626660736:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826654.294605:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826651/real 1787826654]  req@ffff8cd0c8e9f480 x1874671626660864/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826676 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826654.294624:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626660864/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826676 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826654.294632:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826654.294636:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826654.294639:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626660864:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826654.486238:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826654.870211:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826655.510202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826655.894276:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787826656.342468:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826656.342478:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787826656.342482:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826656.342491:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 542
00000100:00080000:0.0:1787826656.342502:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826656.342626:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826656.342661:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626660992/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826656.342694:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826656.342709:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826656.342711:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826656.342715:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 542
00000100:00080000:0.0:1787826656.342720:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826656.342724:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826656.342733:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826656.343661:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626660992:192.168.203.145@tcp:250:kworker.0
00000800:00000100:3.0:1787826656.343732:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826656.343777:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826656/real 1787826656]  req@ffff8cd0c8e9c000 x1874671626660992/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826681 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826656.343793:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626660992/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826681 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826656.343809:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826656.343813:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826656.343817:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626660992:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826656.344863:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626661120/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826656.344892:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826656.344896:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787826656.344899:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826656.344904:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 542
00000100:00080000:0.0:1787826656.344911:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826656.344919:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826656.344939:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826656.344939:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626661120:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826656.344963:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626661248/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826656.344974:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826656.344978:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826656.344980:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826656.344984:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 542
00000100:00080000:0.0:1787826656.344988:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1787826656.344992:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787826656.344996:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826656.345016:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826656.345016:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826656/real 1787826656]  req@ffff8cd0c8e9f840 x1874671626661120/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826681 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826656.345033:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626661376/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826656.345073:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626661120/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826681 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826656.345114:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826656.345119:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826656.345122:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626661120:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826656.345140:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626661248:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826656.345163:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826656.345174:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826656/real 1787826656]  req@ffff8cd0c8e9ed00 x1874671626661248/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826681 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826656.345185:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626661248/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826681 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826656.345191:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826656.345194:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826656.345197:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626661248:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826656.345202:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626661376:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826656.345219:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826656.345228:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826656/real 1787826656]  req@ffff8cd0c8e9d680 x1874671626661376/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826681 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826656.345238:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626661376/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826681 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826656.345255:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826656.345258:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826656.345261:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626661376:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826656.534198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826656.918333:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826657.558189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826657.942280:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826658.582192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826658.966272:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826659.606204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826659.990334:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826660.630204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826661.014432:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787826661.462237:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826661.462250:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787826661.462255:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826661.462267:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 547
00000100:00080000:0.0:1787826661.462279:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826661.462285:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826661.462326:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626661504/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826661.462353:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826661.462359:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826661.462360:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826661.462363:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 547
00000100:00080000:0.0:1787826661.462365:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826661.462368:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826661.462377:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826661.462385:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626661632/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826661.462392:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787826661.462408:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626661504:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787826661.462410:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787826661.462413:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826661.462418:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 547
00000100:00080000:0.0:1787826661.462421:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826661.462423:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826661.462428:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826661.462435:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626661760/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826661.462442:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826661.462445:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826661.462446:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826661.462448:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 547
00000100:00080000:0.0:1787826661.462450:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826661.462453:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826661.462455:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826661.462461:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626661888/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787826661.462507:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826661.462566:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826661/real 1787826661]  req@ffff8cd0c8e9de00 x1874671626661504/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826686 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826661.462592:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626661504/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826686 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826661.462603:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826661.462608:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826661.462612:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626661504:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826661.462623:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626661632:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826661.462655:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826661.462668:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826661/real 1787826661]  req@ffff8cd0c8e9e940 x1874671626661632/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826686 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826661.462697:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626661632/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826686 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826661.462703:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826661.462705:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826661.462707:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626661632:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826661.462729:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626661760:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826661.462746:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826661.462753:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826661/real 1787826661]  req@ffff8cd0c8e9e580 x1874671626661760/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826686 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826661.462760:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626661760/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826686 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826661.462765:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826661.462767:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826661.462769:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626661760:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826661.462772:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626661888:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826661.462781:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826661.462786:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826661/real 1787826661]  req@ffff8cd0c8e9cf00 x1874671626661888/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826686 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826661.462806:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626661888/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826686 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826661.462819:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826661.462821:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826661.462823:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626661888:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826661.654223:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826662.038251:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826662.678217:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826663.062229:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826663.702206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826664.086304:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826664.726214:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826665.110352:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826665.750186:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826666.134303:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787826666.582219:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826666.582229:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787826666.582237:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826666.582247:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 552
00000100:00080000:0.0:1787826666.582257:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826666.582279:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826666.582309:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626662016/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826666.582332:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826666.582335:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826666.582337:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826666.582340:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 552
00000100:00080000:0.0:1787826666.582343:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826666.582347:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826666.582355:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826666.582380:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626662016:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826666.582388:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626662144/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826666.582399:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826666.582403:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787826666.582404:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826666.582407:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 552
00000100:00080000:0.0:1787826666.582411:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826666.582415:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826666.582420:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826666.582429:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626662272/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787826666.582438:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787826666.582440:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826666.582444:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826666.582457:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826666.582462:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 552
00000100:00080000:0.0:1787826666.582467:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1787826666.582469:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826666/real 1787826666]  req@ffff8cd0c8e9fc00 x1874671626662016/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826691 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826666.582474:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826666.582484:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826666.582488:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626662016/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826691 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826666.582498:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626662400/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826666.582507:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826666.582515:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826666.582520:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626662016:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826666.582545:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626662144:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826666.582564:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826666.582573:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826666/real 1787826666]  req@ffff8cd0c8e9cf00 x1874671626662144/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826691 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826666.582588:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626662144/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826691 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826666.582593:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826666.582595:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826666.582597:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626662144:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826666.582600:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626662272:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826666.582614:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826666.582620:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826666/real 1787826666]  req@ffff8cd0c8e9e580 x1874671626662272/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826691 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826666.582626:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626662272/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826691 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826666.582631:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826666.582632:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826666.582634:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626662272:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826666.582637:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626662400:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826666.582646:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826666.582651:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826666/real 1787826666]  req@ffff8cd0c8e9e940 x1874671626662400/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826691 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826666.582657:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626662400/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826691 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826666.582668:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826666.582670:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826666.582673:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626662400:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826666.774202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826667.158285:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826667.798188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826668.182282:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826668.822229:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826669.206220:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826669.846191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826670.230582:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826670.870212:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826671.254231:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787826671.702244:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826671.702254:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787826671.702258:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826671.702274:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 557
00000100:00080000:1.0:1787826671.702285:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826671.702292:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787826671.702331:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626662528/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826671.702376:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826671.702379:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826671.702381:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826671.702385:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 557
00000100:00080000:1.0:1787826671.702389:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826671.702393:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826671.702403:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826671.702415:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626662656/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826671.702419:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626662528:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787826671.702426:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826671.702429:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787826671.702430:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826671.702433:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 557
00000100:00080000:1.0:1787826671.702435:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826671.702437:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826671.702441:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826671.702452:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626662784/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826671.702462:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826671.702465:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826671.702467:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826671.702470:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 557
00000100:00080000:1.0:1787826671.702487:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826671.702494:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:3.0:1787826671.702497:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:1.0:1787826671.702500:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826671.702512:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626662912/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826671.702553:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826671/real 1787826671]  req@ffff8cd0d48f4000 x1874671626662528/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826696 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826671.702571:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626662528/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826696 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826671.702580:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826671.702583:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826671.702586:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626662528:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826671.702608:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626662656:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826671.702630:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826671.702643:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826671/real 1787826671]  req@ffff8cd0d48f6580 x1874671626662656/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826696 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826671.702667:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626662656/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826696 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826671.702675:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826671.702677:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826671.702680:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626662656:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826671.702685:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626662784:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826671.702705:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826671.702715:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826671/real 1787826671]  req@ffff8cd0d48f7480 x1874671626662784/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826696 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826671.702741:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626662784/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826696 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826671.702749:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826671.702752:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826671.702755:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626662784:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826671.702769:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626662912:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826671.702788:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826671.702798:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826671/real 1787826671]  req@ffff8cd0d48f4b40 x1874671626662912/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826696 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826671.702808:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626662912/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826696 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826671.702825:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826671.702829:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826671.702831:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626662912:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826671.894835:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826672.278270:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826672.918197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826673.302303:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826673.942233:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826674.326409:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826674.968623:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826675.350200:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826675.990197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826676.374274:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787826676.822272:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826676.822282:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787826676.822286:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826676.822294:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 562
00000100:00080000:2.0:1787826676.822303:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826676.822315:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787826676.822361:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626663040/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826676.822395:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826676.822399:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826676.822409:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826676.822413:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 562
00000100:00080000:2.0:1787826676.822418:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826676.822422:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826676.822432:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826676.822446:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626663168/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826676.822452:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626663040:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787826676.822460:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826676.822464:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787826676.822466:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826676.822470:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 562
00000100:00080000:2.0:1787826676.822473:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826676.822476:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826676.822481:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826676.822492:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626663296/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826676.822500:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826676.822503:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826676.822505:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826676.822509:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 562
00000800:00000100:3.0:1787826676.822519:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787826676.822531:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826676.822537:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826676.822542:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826676.822554:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626663424/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826676.822574:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826676/real 1787826676]  req@ffff8cd0c3da8000 x1874671626663040/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826676.822598:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626663040/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826676.822608:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826676.822612:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826676.822616:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626663040:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826676.822637:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626663168:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826676.822661:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826676.822673:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826676/real 1787826676]  req@ffff8cd0c3da9e00 x1874671626663168/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826676.822708:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626663168/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826676.822716:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826676.822719:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826676.822722:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626663168:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826676.822728:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626663296:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826676.822751:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826676.822762:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826676/real 1787826676]  req@ffff8cd0c3da92c0 x1874671626663296/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826676.822774:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626663296/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826676.822781:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826676.822784:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826676.822787:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626663296:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826676.822792:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626663424:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826676.822810:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826676.822819:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826676/real 1787826676]  req@ffff8cd0c3daa580 x1874671626663424/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826676.822829:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626663424/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826676.822846:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826676.822849:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826676.822852:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626663424:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826677.014314:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826677.398201:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826678.038232:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826678.422264:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826679.062205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826679.446363:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826680.086318:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826680.470300:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826681.111495:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826681.494395:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787826681.942197:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826681.942207:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787826681.942211:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826681.942221:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 567
00000100:00080000:2.0:1787826681.942231:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826681.942237:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787826681.942285:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daad00 x1874671626663552/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826681.942310:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826681.942313:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826681.942315:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826681.942319:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 567
00000100:00080000:2.0:1787826681.942335:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826681.942343:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826681.942352:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826681.942353:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626663552:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787826681.942368:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626663680/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826681.942379:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826681.942403:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787826681.942405:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826681.942410:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 567
00000100:00080000:2.0:1787826681.942414:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1787826681.942416:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787826681.942421:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826681.942428:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826681.942441:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626663808/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826681.942450:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826681.942453:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826681.942455:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826681.942460:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 567
00000100:00100000:3.0:1787826681.942460:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826681/real 1787826681]  req@ffff8cd0c3daad00 x1874671626663552/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787826681.942464:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826681.942472:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826681.942480:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826681.942487:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daad00 x1874671626663552/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787826681.942493:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626663936/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826681.942498:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826681.942505:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826681.942509:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626663552:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826681.942527:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626663680:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826681.942544:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826681.942552:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826681/real 1787826681]  req@ffff8cd0c3daa580 x1874671626663680/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826681.942569:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626663680/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826681.942574:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826681.942576:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826681.942578:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626663680:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826681.942582:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626663808:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826681.942595:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826681.942601:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826681/real 1787826681]  req@ffff8cd0c3da92c0 x1874671626663808/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826681.942612:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626663808/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826681.942618:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826681.942621:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826681.942624:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626663808:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826681.942630:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626663936:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826681.942660:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826681.942675:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826681/real 1787826681]  req@ffff8cd0c3da9e00 x1874671626663936/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826681.942686:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626663936/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826681.942703:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826681.942706:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826681.942709:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626663936:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826682.134203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826682.518324:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826683.158231:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826683.542316:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826684.182214:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826684.566284:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826685.206246:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826685.590310:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826686.230196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826686.614238:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787826687.062191:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826687.062198:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787826687.062201:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826687.062208:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 572
00000100:00080000:0.0:1787826687.062217:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826687.062222:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826687.062256:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626664064/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826687.062278:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826687.062301:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826687.062304:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826687.062308:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 572
00000100:00100000:3.0:1787826687.062312:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626664064:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787826687.062314:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826687.062320:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826687.062328:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826687.062346:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626664192/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826687.062358:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826687.062361:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000800:00000100:3.0:1787826687.062362:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787826687.062364:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826687.062368:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 572
00000100:00080000:0.0:1787826687.062371:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826687.062374:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826687.062378:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826687.062388:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626664320/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826687.062397:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826687/real 1787826687]  req@ffff8cd0c8e9de00 x1874671626664064/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826712 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826687.062399:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826687.062408:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:3.0:1787826687.062415:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626664064/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826712 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826687.062416:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826687.062426:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 572
00000100:00080000:3.0:1787826687.062428:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826687.062432:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826687.062432:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826687.062436:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626664064:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787826687.062444:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826687.062457:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826687.062457:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626664192:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826687.062475:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826687.062483:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626664448/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826687.062486:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826687/real 1787826687]  req@ffff8cd0c8e9d680 x1874671626664192/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826712 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826687.062509:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626664192/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826712 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826687.062517:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826687.062522:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826687.062525:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626664192:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826687.062530:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626664320:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826687.062552:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826687.062562:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826687/real 1787826687]  req@ffff8cd0c8e9ed00 x1874671626664320/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826712 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826687.062572:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626664320/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826712 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826687.062579:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826687.062582:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826687.062592:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626664320:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826687.062609:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626664448:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826687.062641:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826687.062652:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826687/real 1787826687]  req@ffff8cd0c8e9f840 x1874671626664448/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826712 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826687.062663:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626664448/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826712 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826687.062680:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826687.062684:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826687.062687:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626664448:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826687.254173:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826687.638219:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826688.278190:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826688.662276:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826689.302181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826689.686208:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826690.326194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826690.710248:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826691.350157:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826691.734200:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787826692.182198:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826692.182207:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787826692.182210:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826692.182219:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 578
00000100:00080000:0.0:1787826692.182230:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826692.182237:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826692.182270:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626664576/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826692.182290:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826692.182292:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826692.182294:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826692.182296:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 578
00000100:00080000:0.0:1787826692.182298:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826692.182301:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826692.182308:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826692.182318:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626664704/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826692.182324:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826692.182337:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787826692.182338:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826692.182341:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 578
00000100:00080000:0.0:1787826692.182343:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1787826692.182347:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626664576:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787826692.182348:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826692.182354:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826692.182361:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626664832/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826692.182368:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826692.182370:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826692.182371:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826692.182373:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 578
00000100:00080000:0.0:1787826692.182375:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826692.182377:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826692.182380:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826692.182387:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626664960/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787826692.182415:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826692.182455:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826692/real 1787826692]  req@ffff8cd0c8e9c000 x1874671626664576/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826717 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826692.182471:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626664576/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826717 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826692.182481:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826692.182485:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826692.182489:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626664576:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826692.182496:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626664704:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826692.182515:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826692.182525:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826692/real 1787826692]  req@ffff8cd0c8e9f480 x1874671626664704/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826717 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826692.182550:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626664704/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826717 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826692.182559:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826692.182562:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826692.182565:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626664704:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826692.182601:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626664832:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826692.182622:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826692.182639:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826692/real 1787826692]  req@ffff8cd0c8e9cb40 x1874671626664832/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826717 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826692.182651:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626664832/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826717 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826692.182659:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826692.182663:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826692.182666:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626664832:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826692.182672:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626664960:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826692.182688:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826692.182697:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826692/real 1787826692]  req@ffff8cd0c8e9c3c0 x1874671626664960/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826717 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826692.182708:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626664960/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826717 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826692.182727:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826692.182730:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826692.182733:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626664960:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826692.374206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826692.758229:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826693.398204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826693.782260:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826694.422180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826694.807623:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826695.446201:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826695.830340:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826696.470201:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826696.854299:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787826697.302183:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826697.302198:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787826697.302204:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826697.302218:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 583
00000100:00080000:0.0:1787826697.302233:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826697.302263:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826697.302306:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d2c0 x1874671626665088/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826697.302338:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826697.302342:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826697.302343:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826697.302347:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 583
00000100:00080000:0.0:1787826697.302351:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826697.302355:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826697.302367:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826697.302382:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626665216/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826697.302393:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826697.302396:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787826697.302398:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826697.302402:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 583
00000100:00080000:0.0:1787826697.302405:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826697.302409:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826697.302415:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826697.302422:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626665088:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826697.302434:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626665344/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826697.302448:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826697.302451:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826697.302459:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826697.302463:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 583
00000100:00080000:0.0:1787826697.302469:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826697.302473:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826697.302479:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826697.302491:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626665472/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787826697.302494:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826697.302558:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826697/real 1787826697]  req@ffff8cd0c8e9d2c0 x1874671626665088/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826722 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826697.302574:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d2c0 x1874671626665088/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826722 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826697.302581:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826697.302584:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826697.302587:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626665088:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826697.302592:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626665216:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826697.302610:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826697.302618:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826697/real 1787826697]  req@ffff8cd0c8e9c3c0 x1874671626665216/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826722 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826697.302641:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626665216/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826722 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826697.302649:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826697.302652:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826697.302655:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626665216:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826697.302693:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626665344:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826697.302715:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826697.302726:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826697/real 1787826697]  req@ffff8cd0c8e9cb40 x1874671626665344/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826722 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826697.302736:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626665344/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826722 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826697.302752:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826697.302756:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826697.302759:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626665344:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826697.302765:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626665472:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826697.302783:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826697.302793:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826697/real 1787826697]  req@ffff8cd0c8e9f480 x1874671626665472/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826722 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826697.302805:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626665472/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826722 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826697.302823:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826697.302827:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826697.302830:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626665472:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826697.494222:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826697.878216:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826698.518205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826698.902230:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826699.542194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826699.926224:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826700.566208:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787826700.950269:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826701.590202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826701.974293:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787826702.422245:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826702.422259:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787826702.422263:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826702.422276:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 588
00000100:00080000:0.0:1787826702.422290:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826702.422299:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826702.422357:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626665600/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826702.422394:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826702.422398:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826702.422400:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826702.422404:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 588
00000100:00080000:0.0:1787826702.422408:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826702.422412:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826702.422424:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826702.422450:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626665728/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826702.422457:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826702.422459:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787826702.422461:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826702.422463:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 588
00000100:00080000:0.0:1787826702.422466:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826702.422469:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826702.422473:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826702.422482:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626665856/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826702.422493:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787826702.422498:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626665600:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787826702.422499:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826702.422502:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826702.422519:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 588
00000100:00080000:0.0:1787826702.422524:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826702.422528:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826702.422533:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826702.422545:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626665984/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787826702.422590:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826702.422635:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826702/real 1787826702]  req@ffff8cd0c8e9c000 x1874671626665600/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826727 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826702.422648:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626665600/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826727 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826702.422657:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826702.422660:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826702.422662:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626665600:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826702.422667:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626665728:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826702.422687:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826702.422698:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826702/real 1787826702]  req@ffff8cd0c8e9f840 x1874671626665728/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826727 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826702.422724:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626665728/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826727 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826702.422731:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826702.422735:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826702.422737:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626665728:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826702.422766:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626665856:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826702.422789:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826702.422799:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826702/real 1787826702]  req@ffff8cd0c8e9ed00 x1874671626665856/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826727 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826702.422811:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626665856/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826727 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826702.422818:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826702.422821:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826702.422824:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626665856:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826702.422830:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626665984:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826702.422847:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826702.422857:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826702/real 1787826702]  req@ffff8cd0c8e9d680 x1874671626665984/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826727 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826702.422866:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626665984/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826727 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826702.422886:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826702.422890:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826702.422893:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626665984:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826702.614212:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826702.998317:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826703.638178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826704.022246:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826704.662157:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826705.046225:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826705.686173:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826706.070206:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826706.710175:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826707.094261:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787826707.542249:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826707.542261:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787826707.542266:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826707.542276:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 593
00000100:00080000:2.0:1787826707.542287:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826707.542294:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787826707.542347:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626666112/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826707.542385:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826707.542388:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826707.542389:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826707.542392:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 593
00000100:00080000:2.0:1787826707.542395:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826707.542398:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826707.542407:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826707.542417:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626666240/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826707.542431:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826707.542434:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787826707.542435:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826707.542439:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 593
00000100:00080000:2.0:1787826707.542442:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826707.542444:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826707.542449:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826707.542460:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626666368/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826707.542460:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626666112:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787826707.542494:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826707.542498:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826707.542500:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826707.542506:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 593
00000100:00080000:2.0:1787826707.542516:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826707.542520:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826707.542525:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826707.542534:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626666496/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787826707.542545:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826707.542585:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826707/real 1787826707]  req@ffff8cd0c3da8000 x1874671626666112/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826732 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826707.542603:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626666112/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826732 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826707.542613:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826707.542617:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826707.542621:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626666112:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826707.542629:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626666240:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826707.542648:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826707.542656:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826707/real 1787826707]  req@ffff8cd0c3dab840 x1874671626666240/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826732 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826707.542663:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626666240/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826732 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826707.542683:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826707.542687:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826707.542690:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626666240:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826707.542714:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626666368:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826707.542736:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826707.542749:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826707/real 1787826707]  req@ffff8cd0c3da8780 x1874671626666368/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826732 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826707.542760:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626666368/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826732 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826707.542768:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826707.542771:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826707.542774:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626666368:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826707.542780:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626666496:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826707.542801:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826707.542813:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826707/real 1787826707]  req@ffff8cd0c3dabc00 x1874671626666496/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826732 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826707.542824:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626666496/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826732 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826707.542832:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826707.542835:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826707.542845:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626666496:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826707.734163:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826708.118245:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826708.758168:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826709.142514:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826709.782208:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826710.166212:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826710.806235:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826711.190254:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826711.830224:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826712.214239:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787826712.662184:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826712.662192:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787826712.662196:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826712.662203:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 598
00000100:00080000:2.0:1787826712.662211:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826712.662216:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787826712.662253:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626666624/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826712.662279:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826712.662282:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826712.662284:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826712.662287:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 598
00000100:00080000:2.0:1787826712.662290:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826712.662293:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826712.662303:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826712.662316:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626666752/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826712.662324:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626666624:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787826712.662328:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826712.662346:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787826712.662348:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826712.662353:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 598
00000100:00080000:2.0:1787826712.662366:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826712.662371:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:3.0:1787826712.662373:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:2.0:1787826712.662379:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826712.662393:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626666880/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826712.662403:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826712.662406:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826712.662410:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1787826712.662411:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826712/real 1787826712]  req@ffff8cd0c3daa1c0 x1874671626666624/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826737 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787826712.662417:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 598
00000100:00080000:2.0:1787826712.662424:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826712.662430:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826712.662445:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826712.662446:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626666624/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826737 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787826712.662459:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626667008/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826712.662460:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826712.662469:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826712.662476:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626666624:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826712.662498:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626666752:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826712.662515:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826712.662523:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826712/real 1787826712]  req@ffff8cd0c3da9a40 x1874671626666752/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826737 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826712.662531:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626666752/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826737 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826712.662536:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826712.662545:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826712.662555:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626666752:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826712.662560:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626666880:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826712.662580:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826712.662591:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826712/real 1787826712]  req@ffff8cd0c3dab0c0 x1874671626666880/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826737 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826712.662602:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626666880/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826737 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826712.662611:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826712.662614:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826712.662617:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626666880:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826712.662622:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626667008:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826712.662635:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826712.662642:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826712/real 1787826712]  req@ffff8cd0c3daa940 x1874671626667008/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826737 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826712.662648:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626667008/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826737 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826712.662656:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826712.662659:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826712.662661:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626667008:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826712.854209:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826713.238283:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826713.878193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826714.262238:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826714.902197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826715.286251:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826715.926234:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826716.310253:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826716.950206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826717.334250:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787826717.782266:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826717.782277:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787826717.782281:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826717.782290:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 603
00000100:00080000:0.0:1787826717.782301:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826717.782308:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826717.782350:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626667136/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826717.782381:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826717.782384:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826717.782386:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826717.782412:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 603
00000100:00080000:0.0:1787826717.782417:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826717.782421:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826717.782431:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826717.782444:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626667136:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826717.782446:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626667264/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826717.782471:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826717.782474:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787826717.782476:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826717.782480:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 603
00000100:00080000:0.0:1787826717.782485:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826717.782489:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826717.782494:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826717.782508:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626667392/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826717.782517:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826717.782525:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826717.782526:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826717.782530:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 603
00000100:00080000:0.0:1787826717.782535:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826717.782539:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826717.782545:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826717.782546:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626667264:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826717.782560:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626667520/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826717.782586:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626667392:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826717.782615:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626667520:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826717.974228:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826718.358283:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826718.998222:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826719.382260:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826720.023520:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826720.406215:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:2.0:1787826720.854497:0:2346:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:2.0:1787826720.854535:0:2346:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:2.0:1787826720.854560:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787826720.854598:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787826720.854606:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787826720.854613:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:3.0:1787826720.854925:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826717/real 1787826720]  req@ffff8cd0c8e9cf00 x1874671626667520/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826720.854950:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626667520/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826720.854961:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826720.854967:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826720.854971:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626667520:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826720.854978:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826717/real 1787826720]  req@ffff8cd0c8e9e940 x1874671626667264/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826720.854990:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626667264/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826720.854997:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826720.855012:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826720.855016:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626667264:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826720.855022:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826717/real 1787826720]  req@ffff8cd0c8e9e580 x1874671626667392/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826720.855034:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626667392/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826720.855041:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826720.855045:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826720.855048:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626667392:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826720.855053:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826717/real 1787826720]  req@ffff8cd0c8e9de00 x1874671626667136/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826720.855064:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626667136/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826720.855071:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826720.855074:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826720.855077:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626667136:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826721.046262:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826721.430561:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826722.070226:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826722.454266:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787826722.902223:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826722.902232:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787826722.902236:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826722.902246:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 608
00000100:00080000:1.0:1787826722.902255:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826722.902262:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787826722.902298:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626667648/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826722.902327:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826722.902332:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826722.902335:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826722.902340:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 608
00000100:00080000:1.0:1787826722.902345:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826722.902348:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826722.902357:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826722.902381:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626667776/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826722.902392:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1787826722.902403:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626667648:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787826722.902414:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787826722.902416:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826722.902421:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 608
00000100:00080000:1.0:1787826722.902425:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826722.902429:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826722.902435:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826722.902448:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626667904/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826722.902458:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826722.902461:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826722.902463:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826722.902469:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 608
00000800:00000100:0.0:1787826722.902471:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787826722.902474:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826722.902479:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826722.902483:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826722.902494:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626668032/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826722.902517:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826722/real 1787826722]  req@ffff8cd0d48f4b40 x1874671626667648/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826722.902534:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626667648/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826722.902544:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826722.902547:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826722.902551:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626667648:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826722.902573:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626667776:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826722.902595:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826722.902606:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826722/real 1787826722]  req@ffff8cd0d48f7480 x1874671626667776/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826722.902616:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626667776/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826722.902624:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826722.902627:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826722.902630:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626667776:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826722.902635:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626667904:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787826722.902653:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826722.902663:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826722/real 1787826722]  req@ffff8cd0d48f6580 x1874671626667904/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826722.902688:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626667904/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826722.902695:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826722.902698:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826722.902701:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626667904:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826722.902707:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626668032:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826722.902727:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826722.902738:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826722/real 1787826722]  req@ffff8cd0d48f4000 x1874671626668032/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826722.902749:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626668032/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826722.902757:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826722.902760:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826722.902763:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626668032:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826723.094248:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826723.478330:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826724.118207:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826724.502218:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826725.142240:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826725.526428:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826726.166195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826726.550240:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826727.190178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826727.574230:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787826728.022219:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826728.022230:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787826728.022234:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826728.022244:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 613
00000100:00080000:1.0:1787826728.022255:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826728.022262:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787826728.022301:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626668160/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826728.022361:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826728.022365:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826728.022367:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826728.022372:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 613
00000100:00080000:1.0:1787826728.022375:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826728.022378:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826728.022389:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826728.022400:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626668288/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826728.022408:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826728.022411:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787826728.022413:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826728.022415:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 613
00000100:00080000:1.0:1787826728.022417:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826728.022420:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826728.022423:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826728.022432:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626668416/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826728.022443:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626668160:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787826728.022445:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826728.022460:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826728.022463:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826728.022468:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 613
00000100:00080000:1.0:1787826728.022475:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826728.022480:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826728.022484:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826728.022494:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626668544/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787826728.022523:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826728.022568:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826728/real 1787826728]  req@ffff8cd0d48f7840 x1874671626668160/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826753 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826728.022596:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626668160/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826753 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826728.022604:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826728.022607:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826728.022610:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626668160:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826728.022615:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626668288:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826728.022633:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826728.022647:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826728/real 1787826728]  req@ffff8cd0d48f5a40 x1874671626668288/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826753 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826728.022654:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626668288/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826753 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826728.022660:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826728.022663:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826728.022665:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626668288:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826728.022687:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626668416:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787826728.022699:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826728.022706:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826728/real 1787826728]  req@ffff8cd0d48f43c0 x1874671626668416/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826753 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826728.022717:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626668416/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826753 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826728.022736:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826728.022739:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826728.022742:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626668416:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826728.022749:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626668544:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826728.022772:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826728.022784:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826728/real 1787826728]  req@ffff8cd0d48f70c0 x1874671626668544/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826753 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826728.022793:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626668544/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826753 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826728.022798:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826728.022800:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826728.022802:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626668544:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826728.214221:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826728.598240:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826729.238411:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826729.622185:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826730.262204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826730.646237:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826731.286197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826731.670213:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826732.310157:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826732.694250:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787826733.142250:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826733.142259:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787826733.142263:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826733.142272:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 618
00000100:00080000:0.0:1787826733.142283:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826733.142290:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826733.142346:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626668672/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826733.142369:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826733.142372:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826733.142374:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826733.142377:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 618
00000100:00080000:0.0:1787826733.142382:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826733.142386:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826733.142397:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826733.142410:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626668800/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826733.142421:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787826733.142422:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626668672:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787826733.142437:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787826733.142440:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826733.142445:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 618
00000100:00080000:0.0:1787826733.142449:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826733.142453:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826733.142458:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826733.142469:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626668928/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826733.142479:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826733.142482:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826733.142484:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:3.0:1787826733.142489:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787826733.142498:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 618
00000100:00080000:0.0:1787826733.142502:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826733.142507:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826733.142512:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826733.142524:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626669056/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826733.142535:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826733/real 1787826733]  req@ffff8cd0c8e9fc00 x1874671626668672/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826758 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826733.142554:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626668672/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826758 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826733.142566:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826733.142569:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826733.142573:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626668672:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826733.142595:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626668800:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826733.142631:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826733.142643:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826733/real 1787826733]  req@ffff8cd0c8e9de00 x1874671626668800/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826758 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826733.142654:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626668800/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826758 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826733.142663:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826733.142666:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826733.142669:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626668800:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826733.142675:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626668928:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826733.142704:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826733.142715:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826733/real 1787826733]  req@ffff8cd0c8e9e580 x1874671626668928/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826758 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826733.142726:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626668928/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826758 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826733.142734:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826733.142737:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826733.142740:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626668928:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826733.142745:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626669056:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826733.142764:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826733.142774:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826733/real 1787826733]  req@ffff8cd0c8e9e940 x1874671626669056/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826758 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826733.142785:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626669056/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826758 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826733.142793:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826733.142796:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826733.142799:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626669056:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826733.334201:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826733.718164:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826734.359523:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826734.742270:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826735.382220:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826735.766266:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826736.406201:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826736.790251:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826737.430189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826737.814243:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787826738.262200:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826738.262210:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787826738.262214:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826738.262223:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 624
00000100:00080000:0.0:1787826738.262238:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826738.262244:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826738.262289:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626669184/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826738.262318:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826738.262321:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826738.262323:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826738.262326:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 624
00000100:00080000:0.0:1787826738.262329:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826738.262332:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826738.262354:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826738.262367:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626669312/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826738.262374:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626669184:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787826738.262381:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826738.262390:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787826738.262391:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826738.262396:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 624
00000100:00080000:0.0:1787826738.262412:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826738.262417:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826738.262422:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787826738.262436:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826738.262437:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626669440/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826738.262449:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826738.262452:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826738.262454:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826738.262458:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 624
00000100:00080000:0.0:1787826738.262462:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826738.262466:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826738.262471:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826738.262473:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826738/real 1787826738]  req@ffff8cd0c8e9cf00 x1874671626669184/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826763 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826738.262485:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626669568/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826738.262505:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626669184/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826763 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826738.262514:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826738.262517:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826738.262521:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626669184:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826738.262545:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626669312:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826738.262568:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826738.262581:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826738/real 1787826738]  req@ffff8cd0c8e9d680 x1874671626669312/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826763 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826738.262593:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626669312/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826763 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826738.262600:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826738.262603:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826738.262606:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626669312:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826738.262612:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626669440:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826738.262632:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826738.262642:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826738/real 1787826738]  req@ffff8cd0c8e9ed00 x1874671626669440/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826763 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826738.262671:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626669440/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826763 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826738.262679:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826738.262686:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826738.262689:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626669440:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826738.262695:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626669568:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826738.262716:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826738.262725:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826738/real 1787826738]  req@ffff8cd0c8e9f840 x1874671626669568/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826763 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826738.262733:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626669568/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826763 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826738.262738:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826738.262740:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826738.262742:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626669568:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826738.454245:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826738.838278:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826739.478197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826739.862802:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826740.502178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826740.886169:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826741.526196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826741.910269:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826742.550291:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826742.935311:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787826743.382197:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826743.382204:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787826743.382207:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826743.382214:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 629
00000100:00080000:0.0:1787826743.382234:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826743.382240:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826743.382271:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626669696/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826743.382298:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826743.382301:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826743.382303:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826743.382306:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 629
00000100:00080000:0.0:1787826743.382310:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826743.382313:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826743.382323:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826743.382337:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626669824/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826743.382340:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626669696:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787826743.382355:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826743.382358:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787826743.382360:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826743.382365:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 629
00000100:00080000:0.0:1787826743.382369:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826743.382386:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826743.382391:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787826743.382395:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826743.382402:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626669952/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826743.382413:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826743.382417:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826743.382418:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826743.382422:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 629
00000100:00080000:0.0:1787826743.382427:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826743.382431:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826743.382437:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826743.382438:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826743/real 1787826743]  req@ffff8cd0c8e9c000 x1874671626669696/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826768 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826743.382449:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626670080/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826743.382475:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626669696/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826768 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826743.382483:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826743.382486:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826743.382489:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626669696:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826743.382507:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626669824:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826743.382523:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826743.382531:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826743/real 1787826743]  req@ffff8cd0c8e9f480 x1874671626669824/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826768 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826743.382538:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626669824/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826768 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826743.382546:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826743.382549:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826743.382552:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626669824:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826743.382557:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626669952:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826743.382577:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826743.382587:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826743/real 1787826743]  req@ffff8cd0c8e9cb40 x1874671626669952/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826768 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826743.382598:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626669952/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826768 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826743.382616:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826743.382619:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826743.382622:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626669952:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826743.382628:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626670080:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826743.382647:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826743.382657:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826743/real 1787826743]  req@ffff8cd0c8e9c3c0 x1874671626670080/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826768 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826743.382668:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626670080/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826768 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826743.382675:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826743.382677:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826743.382679:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626670080:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826743.574198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826743.958296:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826744.598178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826744.982262:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826745.622223:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826746.006265:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826746.646401:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826747.030290:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826747.670221:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826748.054264:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787826748.502892:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826748.502901:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787826748.502905:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826748.502914:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 634
00000100:00080000:1.0:1787826748.502934:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826748.502942:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787826748.502977:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626670208/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826748.503007:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826748.503011:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826748.503012:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826748.503015:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 634
00000100:00080000:1.0:1787826748.503019:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826748.503036:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826748.503045:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826748.503055:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626670336/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826748.503064:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826748.503081:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787826748.503083:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826748.503106:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 634
00000100:00080000:1.0:1787826748.503110:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826748.503115:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826748.503120:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826748.503131:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626670464/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826748.503141:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826748.503144:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826748.503146:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826748.503149:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 634
00000100:00080000:1.0:1787826748.503152:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826748.503155:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826748.503160:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826748.503170:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626670592/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826748.503244:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626670208:192.168.203.145@tcp:250:kworker.0
00000800:00000100:3.0:1787826748.503304:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826748.503341:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826748/real 1787826748]  req@ffff8cd0d48f61c0 x1874671626670208/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826773 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826748.503359:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626670208/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826773 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826748.503376:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826748.503380:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826748.503383:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626670208:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826748.503390:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626670336:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826748.503415:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826748.503431:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826748/real 1787826748]  req@ffff8cd0d48f7c00 x1874671626670336/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826773 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826748.503442:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626670336/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826773 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826748.503449:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826748.503452:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826748.503454:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626670336:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826748.503460:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626670464:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826748.503478:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826748.503488:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826748/real 1787826748]  req@ffff8cd0d48f52c0 x1874671626670464/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826773 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826748.503497:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626670464/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826773 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826748.503503:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826748.503506:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826748.503508:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626670464:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826748.503520:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626670592:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826748.503539:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826748.503548:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826748/real 1787826748]  req@ffff8cd0d48f6d00 x1874671626670592/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826773 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826748.503558:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626670592/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826773 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826748.503564:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826748.503567:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826748.503569:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626670592:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826748.694208:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826749.078226:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826749.718214:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826750.102305:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826750.742253:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826751.126242:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826751.766256:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826752.150288:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826752.790242:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826753.174339:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787826753.622643:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826753.622653:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787826753.622658:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826753.622667:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 639
00000100:00080000:1.0:1787826753.622677:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826753.622684:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787826753.622718:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626670720/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826753.622751:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826753.622754:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826753.622755:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826753.622757:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 639
00000100:00080000:1.0:1787826753.622760:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826753.622763:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826753.622770:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826753.622779:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626670848/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826753.622789:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826753.622792:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787826753.622794:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826753.622798:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 639
00000100:00080000:1.0:1787826753.622801:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826753.622804:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826753.622809:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826753.622815:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626670720:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787826753.622823:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626670976/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826753.622833:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826753.622837:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826753.622838:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826753.622842:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 639
00000100:00080000:1.0:1787826753.622852:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826753.622856:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826753.622862:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826753.622873:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626671104/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787826753.622887:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826753.622929:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826753/real 1787826753]  req@ffff8cd0d48f6d00 x1874671626670720/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826778 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826753.622946:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626670720/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826778 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826753.622956:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826753.622960:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826753.622964:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626670720:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826753.622989:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626670848:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826753.623014:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826753.623027:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826753/real 1787826753]  req@ffff8cd0d48f52c0 x1874671626670848/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826778 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826753.623038:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626670848/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826778 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826753.623046:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826753.623049:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826753.623052:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626670848:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826753.623106:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626670976:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826753.623129:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826753.623138:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826753/real 1787826753]  req@ffff8cd0d48f7c00 x1874671626670976/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826778 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826753.623145:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626670976/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826778 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826753.623162:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826753.623166:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826753.623172:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626670976:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826753.623178:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626671104:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826753.623212:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826753.623224:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826753/real 1787826753]  req@ffff8cd0d48f61c0 x1874671626671104/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826778 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826753.623245:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626671104/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826778 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826753.623253:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826753.623256:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826753.623260:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626671104:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826753.814188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826754.198221:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826754.838186:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826755.222207:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826755.862209:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826756.246314:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826756.886210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826757.270258:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826757.910252:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826758.294291:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787826758.742256:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826758.742270:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787826758.742275:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826758.742288:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 644
00000100:00080000:0.0:1787826758.742302:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826758.742311:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826758.742361:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d2c0 x1874671626671232/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826758.742394:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826758.742396:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826758.742398:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826758.742401:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 644
00000100:00080000:0.0:1787826758.742406:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826758.742414:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826758.742425:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826758.742442:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626671360/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826758.742452:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826758.742455:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787826758.742457:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826758.742461:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 644
00000100:00080000:0.0:1787826758.742464:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826758.742467:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826758.742492:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826758.742505:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626671488/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826758.742517:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826758.742520:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826758.742522:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826758.742528:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 644
00000100:00100000:3.0:1787826758.742528:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626671232:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787826758.742536:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826758.742540:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826758.742544:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826758.742553:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626671616/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787826758.742615:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826758.742663:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826758/real 1787826758]  req@ffff8cd0c8e9d2c0 x1874671626671232/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826783 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826758.742680:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d2c0 x1874671626671232/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826783 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826758.742691:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826758.742696:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826758.742700:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626671232:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826758.742707:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626671360:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826758.742731:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826758.742744:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826758/real 1787826758]  req@ffff8cd0c8e9c3c0 x1874671626671360/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826783 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826758.742764:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626671360/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826783 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826758.742772:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826758.742775:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826758.742778:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626671360:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826758.742809:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626671488:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826758.742832:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826758.742843:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826758/real 1787826758]  req@ffff8cd0c8e9cb40 x1874671626671488/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826783 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826758.742855:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626671488/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826783 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826758.742862:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826758.742866:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826758.742869:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626671488:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826758.742874:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626671616:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826758.742893:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826758.742904:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826758/real 1787826758]  req@ffff8cd0c8e9f480 x1874671626671616/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826783 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826758.742912:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626671616/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826783 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826758.742938:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826758.742942:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826758.742945:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626671616:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826758.934257:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826759.318310:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826759.958208:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826760.342222:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826760.982210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826761.366234:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826762.006212:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826762.390229:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826763.030214:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826763.414228:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787826763.862198:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826763.862207:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787826763.862211:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826763.862221:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 649
00000100:00080000:0.0:1787826763.862232:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826763.862239:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826763.862271:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626671744/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826763.862312:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826763.862315:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826763.862316:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826763.862319:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 649
00000100:00080000:0.0:1787826763.862322:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826763.862325:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826763.862332:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826763.862342:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626671872/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826763.862351:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826763.862353:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787826763.862354:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826763.862358:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 649
00000100:00100000:3.0:1787826763.862359:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626671744:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787826763.862361:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826763.862366:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826763.862372:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826763.862379:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626672000/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826763.862385:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826763.862387:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826763.862388:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826763.862390:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 649
00000100:00080000:0.0:1787826763.862409:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826763.862415:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826763.862421:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787826763.862430:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826763.862436:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626672128/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826763.862471:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826763/real 1787826763]  req@ffff8cd0c8e9c000 x1874671626671744/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826763.862489:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626671744/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826763.862500:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826763.862505:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826763.862509:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626671744:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826763.862532:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626671872:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826763.862554:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826763.862565:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826763/real 1787826763]  req@ffff8cd0c8e9f840 x1874671626671872/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826763.862577:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626671872/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826763.862591:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826763.862594:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826763.862598:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626671872:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826763.862604:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626672000:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826763.862624:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826763.862635:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826763/real 1787826763]  req@ffff8cd0c8e9ed00 x1874671626672000/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826763.862645:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626672000/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826763.862653:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826763.862656:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826763.862659:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626672000:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826763.862664:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626672128:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826763.862682:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826763.862693:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826763/real 1787826763]  req@ffff8cd0c8e9d680 x1874671626672128/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826763.862704:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626672128/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826763.862711:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826763.862714:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826763.862723:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626672128:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826764.054199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826764.438171:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826765.078213:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826765.462220:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826766.102256:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826766.486257:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826767.126287:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826767.510228:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826768.150243:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826768.534231:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787826768.982229:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826768.982239:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787826768.982243:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826768.982253:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 654
00000100:00080000:1.0:1787826768.982263:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826768.982270:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787826768.982300:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626672256/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826768.982319:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826768.982321:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826768.982322:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826768.982325:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 654
00000100:00080000:1.0:1787826768.982327:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826768.982329:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826768.982344:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826768.982352:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626672384/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826768.982359:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787826768.982367:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626672256:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787826768.982375:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787826768.982377:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826768.982380:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 654
00000100:00080000:1.0:1787826768.982383:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826768.982386:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826768.982390:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826768.982400:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626672512/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826768.982409:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826768.982412:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826768.982414:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826768.982418:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 654
00000100:00080000:1.0:1787826768.982422:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826768.982426:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826768.982432:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787826768.982439:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787826768.982444:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626672640/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826768.982485:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826768/real 1787826768]  req@ffff8cd0d48f70c0 x1874671626672256/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826768.982503:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626672256/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826768.982513:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826768.982522:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826768.982526:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626672256:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826768.982548:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626672384:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826768.982568:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826768.982579:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826768/real 1787826768]  req@ffff8cd0d48f43c0 x1874671626672384/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826768.982589:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626672384/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826768.982597:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826768.982600:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826768.982611:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626672384:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826768.982616:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626672512:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826768.982638:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826768.982649:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826768/real 1787826768]  req@ffff8cd0d48f5a40 x1874671626672512/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826768.982659:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626672512/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826768.982667:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826768.982670:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826768.982673:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626672512:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826768.982678:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626672640:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826768.982694:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826768.982703:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826768/real 1787826768]  req@ffff8cd0d48f7840 x1874671626672640/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826768.982721:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626672640/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826768.982729:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826768.982732:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826768.982735:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626672640:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826769.174152:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826769.558224:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826770.198168:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826770.582192:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826771.222143:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826771.606230:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826772.246176:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826772.630243:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826773.270209:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826773.654211:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787826774.102211:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826774.102221:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787826774.102225:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826774.102235:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 659
00000100:00080000:1.0:1787826774.102246:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826774.102252:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787826774.102289:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626672768/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826774.102337:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826774.102341:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826774.102343:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826774.102347:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 659
00000100:00080000:1.0:1787826774.102351:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826774.102355:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826774.102364:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826774.102377:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626672896/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826774.102385:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826774.102388:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787826774.102390:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826774.102405:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 659
00000100:00080000:1.0:1787826774.102409:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826774.102412:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826774.102417:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826774.102427:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626673024/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826774.102437:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826774.102440:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826774.102442:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826774.102446:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 659
00000100:00080000:1.0:1787826774.102456:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1787826774.102462:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626672768:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787826774.102463:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826774.102471:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826774.102484:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626673152/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787826774.102530:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826774.102570:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826774/real 1787826774]  req@ffff8cd0d48f4000 x1874671626672768/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826799 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826774.102587:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626672768/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826799 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826774.102597:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826774.102601:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826774.102604:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626672768:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826774.102611:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626672896:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826774.102634:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826774.102647:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826774/real 1787826774]  req@ffff8cd0d48f6580 x1874671626672896/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826799 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826774.102674:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626672896/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826799 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826774.102682:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826774.102685:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826774.102689:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626672896:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826774.102705:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626673024:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826774.102738:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826774.102762:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826774/real 1787826774]  req@ffff8cd0d48f7480 x1874671626673024/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826799 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826774.102775:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626673024/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826799 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826774.102783:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826774.102786:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826774.102789:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626673024:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826774.102823:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626673152:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826774.102857:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826774.102869:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826774/real 1787826774]  req@ffff8cd0d48f4b40 x1874671626673152/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826799 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826774.102880:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626673152/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826799 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826774.102894:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826774.102898:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826774.102901:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626673152:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826774.294138:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826774.678275:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826775.318193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826775.702222:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826776.342200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826776.726243:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826777.366226:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826777.750238:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826778.390211:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826778.774215:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787826779.222218:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826779.222228:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787826779.222232:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826779.222241:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 665
00000100:00080000:0.0:1787826779.222252:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826779.222258:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826779.222292:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626673280/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826779.222347:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826779.222353:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826779.222355:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826779.222360:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 665
00000100:00080000:0.0:1787826779.222365:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826779.222369:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826779.222379:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826779.222387:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626673280:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826779.222395:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626673408/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826779.222409:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826779.222413:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787826779.222415:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826779.222418:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 665
00000100:00080000:0.0:1787826779.222423:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826779.222427:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826779.222433:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826779.222445:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626673536/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826779.222455:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826779.222459:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826779.222460:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826779.222467:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 665
00000800:00000100:3.0:1787826779.222467:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787826779.222478:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826779.222484:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826779.222489:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826779.222503:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626673664/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826779.222510:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826779/real 1787826779]  req@ffff8cd0c8e9cf00 x1874671626673280/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826804 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826779.222528:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626673280/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826804 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826779.222539:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826779.222544:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826779.222548:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626673280:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826779.222573:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626673408:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826779.222590:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826779.222599:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826779/real 1787826779]  req@ffff8cd0c8e9e940 x1874671626673408/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826804 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826779.222610:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626673408/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826804 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826779.222618:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826779.222621:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826779.222624:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626673408:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826779.222629:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626673536:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826779.222647:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826779.222657:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826779/real 1787826779]  req@ffff8cd0c8e9e580 x1874671626673536/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826804 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826779.222681:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626673536/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826804 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826779.222689:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826779.222692:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826779.222695:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626673536:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826779.222701:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626673664:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826779.222721:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826779.222731:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826779/real 1787826779]  req@ffff8cd0c8e9de00 x1874671626673664/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826804 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826779.222742:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626673664/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826804 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826779.222748:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826779.222752:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826779.222754:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626673664:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826779.414185:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826779.798261:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826780.438245:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826780.822351:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826781.462252:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826781.846895:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826782.486249:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826782.870250:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826783.510246:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826783.894222:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787826784.342230:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826784.342240:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787826784.342245:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826784.342254:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 670
00000100:00080000:1.0:1787826784.342273:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826784.342280:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787826784.342319:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626673792/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826784.342362:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826784.342364:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826784.342366:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826784.342368:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 670
00000100:00080000:1.0:1787826784.342371:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826784.342374:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826784.342381:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826784.342399:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626673920/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826784.342408:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826784.342426:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787826784.342428:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826784.342431:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 670
00000100:00080000:1.0:1787826784.342435:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826784.342439:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826784.342445:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826784.342455:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626674048/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826784.342462:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826784.342465:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826784.342467:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826784.342470:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 670
00000100:00080000:1.0:1787826784.342473:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826784.342477:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826784.342481:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826784.342490:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626674176/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826784.342585:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626673792:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826784.342683:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626673920:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826784.342719:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626674048:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826784.342738:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626674176:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826784.534337:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826784.918237:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826785.558243:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826785.942219:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826786.582207:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826786.966239:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:2.0:1787826787.414840:0:2343:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:2.0:1787826787.414890:0:2343:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:2.0:1787826787.414904:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787826787.414953:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787826787.414961:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787826787.414969:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:3.0:1787826787.415416:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826784/real 1787826787]  req@ffff8cd0d48f4b40 x1874671626673792/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826809 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826787.415440:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626673792/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826809 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826787.415450:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826787.415455:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826787.415459:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626673792:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826787.415466:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826784/real 1787826787]  req@ffff8cd0d48f7480 x1874671626673920/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826809 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826787.415478:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626673920/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826809 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826787.415485:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826787.415488:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826787.415492:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626673920:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826787.415498:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826784/real 1787826787]  req@ffff8cd0d48f6580 x1874671626674048/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826809 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826787.415522:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626674048/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826809 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826787.415527:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826787.415530:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826787.415533:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626674048:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826787.415539:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826784/real 1787826787]  req@ffff8cd0d48f4000 x1874671626674176/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826809 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826787.415549:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626674176/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826809 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826787.415557:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826787.415560:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826787.415563:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626674176:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826787.606207:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826787.990240:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826788.631983:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826789.014277:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787826789.462243:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826789.462249:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787826789.462253:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826789.462261:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 675
00000100:00080000:0.0:1787826789.462271:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826789.462278:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826789.462310:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626674304/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826789.462331:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826789.462335:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826789.462336:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826789.462353:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 675
00000100:00080000:0.0:1787826789.462357:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826789.462361:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826789.462371:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826789.462384:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626674432/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826789.462398:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826789.462401:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787826789.462403:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826789.462406:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 675
00000100:00080000:0.0:1787826789.462409:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826789.462412:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826789.462417:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826789.462426:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626674560/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826789.462433:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626674304:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787826789.462434:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826789.462437:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826789.462439:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826789.462442:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 675
00000100:00080000:0.0:1787826789.462445:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826789.462448:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826789.462451:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826789.462459:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626674688/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787826789.462485:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826789.462522:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826789/real 1787826789]  req@ffff8cd0c8e9fc00 x1874671626674304/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826814 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826789.462538:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626674304/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826814 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826789.462548:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826789.462558:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826789.462562:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626674304:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826789.462570:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626674432:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826789.462594:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826789.462606:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826789/real 1787826789]  req@ffff8cd0c8e9de00 x1874671626674432/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826814 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826789.462616:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626674432/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826814 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826789.462624:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826789.462626:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826789.462629:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626674432:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826789.462654:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626674560:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826789.462673:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826789.462683:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826789/real 1787826789]  req@ffff8cd0c8e9e580 x1874671626674560/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826814 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826789.462693:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626674560/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826814 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826789.462701:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826789.462704:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826789.462707:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626674560:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826789.462712:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626674688:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826789.462738:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826789.462748:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826789/real 1787826789]  req@ffff8cd0c8e9e940 x1874671626674688/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826814 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826789.462758:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626674688/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826814 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826789.462764:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826789.462767:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826789.462770:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626674688:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826789.654203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826790.038234:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826790.678197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826791.062312:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826791.702178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826792.086247:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826792.726195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826793.110551:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826793.750176:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826794.134270:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787826794.582231:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826794.582241:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787826794.582246:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826794.582255:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 680
00000100:00080000:2.0:1787826794.582266:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826794.582273:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787826794.582314:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab480 x1874671626674816/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826794.582339:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826794.582343:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826794.582344:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826794.582348:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 680
00000100:00080000:2.0:1787826794.582352:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826794.582356:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826794.582365:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826794.582378:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626674944/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826794.582389:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826794.582392:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:3.0:1787826794.582402:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626674816:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787826794.582410:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826794.582416:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 680
00000100:00080000:2.0:1787826794.582421:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826794.582425:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826794.582431:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826794.582444:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626675072/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826794.582453:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826794.582457:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826794.582459:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826794.582463:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 680
00000100:00080000:2.0:1787826794.582467:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1787826794.582469:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787826794.582473:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826794.582479:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826794.582491:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626675200/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826794.582524:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826794/real 1787826794]  req@ffff8cd0c3dab480 x1874671626674816/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826819 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826794.582541:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab480 x1874671626674816/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826819 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826794.582556:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826794.582560:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826794.582563:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626674816:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826794.582596:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626674944:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826794.582627:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826794.582638:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826794/real 1787826794]  req@ffff8cd0c3da83c0 x1874671626674944/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826819 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826794.582649:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626674944/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826819 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826794.583118:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826794.583122:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826794.583126:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626674944:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826794.583132:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626675072:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826794.583158:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826794.583169:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826794/real 1787826794]  req@ffff8cd0c3da9680 x1874671626675072/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826819 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826794.583179:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626675072/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826819 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826794.583186:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826794.583190:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826794.583192:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626675072:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826794.583198:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626675200:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826794.583215:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826794.583492:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826794/real 1787826794]  req@ffff8cd0c3da8f00 x1874671626675200/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826819 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826794.583517:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626675200/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826819 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826794.583524:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826794.583527:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826794.583530:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626675200:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826794.774211:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826795.158228:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826795.798227:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826796.182221:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826796.822164:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826797.206256:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826797.846218:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826798.230219:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826798.870205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826799.254244:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787826799.702187:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826799.702195:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787826799.702198:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826799.702211:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 685
00000100:00080000:0.0:1787826799.702221:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826799.702227:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826799.702258:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626675328/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826799.702280:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826799.702283:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826799.702285:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826799.702288:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 685
00000100:00080000:0.0:1787826799.702291:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826799.702294:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826799.702303:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826799.702314:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626675456/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826799.702323:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626675328:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787826799.702343:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826799.702346:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787826799.702348:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826799.702352:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 685
00000100:00080000:0.0:1787826799.702356:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826799.702360:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826799.702366:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826799.702377:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626675584/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826799.702386:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:3.0:1787826799.702387:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787826799.702392:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826799.702394:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826799.702398:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 685
00000100:00080000:0.0:1787826799.702403:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826799.702416:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826799.702421:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826799.702427:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826799/real 1787826799]  req@ffff8cd0c8e9cf00 x1874671626675328/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826824 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826799.702433:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626675712/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826799.702451:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626675328/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826824 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826799.702461:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826799.702466:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826799.702470:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626675328:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826799.702490:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626675456:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826799.702509:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826799.702529:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826799/real 1787826799]  req@ffff8cd0c8e9d680 x1874671626675456/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826824 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826799.702547:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626675456/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826824 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826799.702554:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826799.702557:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826799.702559:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626675456:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826799.702565:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626675584:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826799.702582:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826799.702591:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826799/real 1787826799]  req@ffff8cd0c8e9ed00 x1874671626675584/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826824 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826799.702601:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626675584/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826824 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826799.702607:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826799.702615:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826799.702618:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626675584:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826799.702623:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626675712:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826799.702638:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826799.702647:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826799/real 1787826799]  req@ffff8cd0c8e9f840 x1874671626675712/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826824 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826799.702657:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626675712/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826824 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826799.702674:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826799.702677:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826799.702680:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626675712:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826799.894192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826800.278217:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826800.918206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826801.302260:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826801.942188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826802.326225:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826802.966238:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826803.350262:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826803.990201:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826804.374290:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787826804.822253:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826804.822268:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787826804.822297:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826804.822310:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 690
00000100:00080000:0.0:1787826804.822324:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826804.822333:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826804.822377:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626675840/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826804.822412:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826804.822416:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826804.822418:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826804.822422:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 690
00000100:00080000:0.0:1787826804.822425:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826804.822430:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826804.822443:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826804.822457:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626675968/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826804.822467:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626675840:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787826804.822471:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826804.822478:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787826804.822480:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826804.822484:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 690
00000100:00080000:0.0:1787826804.822488:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826804.822492:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826804.822498:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826804.822507:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626676096/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826804.822534:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826804.822538:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826804.822541:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826804.822545:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 690
00000100:00080000:0.0:1787826804.822550:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1787826804.822553:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787826804.822558:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826804.822564:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826804.822576:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626676224/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826804.822611:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826804/real 1787826804]  req@ffff8cd0c8e9c000 x1874671626675840/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826804.822632:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626675840/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826804.822644:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826804.822649:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826804.822653:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626675840:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826804.822680:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626675968:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826804.822698:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826804.822708:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826804/real 1787826804]  req@ffff8cd0c8e9f480 x1874671626675968/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826804.822715:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626675968/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826804.822727:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826804.822729:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826804.822731:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626675968:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826804.822735:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626676096:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826804.822753:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826804.822762:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826804/real 1787826804]  req@ffff8cd0c8e9cb40 x1874671626676096/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826804.822773:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626676096/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826804.822780:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826804.822783:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826804.822786:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626676096:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826804.822802:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626676224:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826804.822822:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826804.822833:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826804/real 1787826804]  req@ffff8cd0c8e9c3c0 x1874671626676224/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826804.822844:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626676224/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826804.822851:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826804.822854:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826804.822863:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626676224:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826805.014235:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826805.398332:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826806.038248:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826806.422248:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826807.062249:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826807.446246:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826808.086210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826808.470303:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826809.110203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826809.494269:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787826809.942189:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826809.942198:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787826809.942202:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826809.942212:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 695
00000100:00080000:1.0:1787826809.942222:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826809.942229:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787826809.942264:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626676352/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826809.942310:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826809.942313:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826809.942314:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826809.942318:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 695
00000100:00080000:1.0:1787826809.942321:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826809.942324:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826809.942332:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826809.942345:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626676480/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826809.942355:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826809.942358:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787826809.942361:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826809.942364:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 695
00000100:00100000:3.0:1787826809.942371:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626676352:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787826809.942378:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826809.942384:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826809.942390:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826809.942401:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626676608/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826809.942412:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826809.942415:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826809.942417:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826809.942420:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 695
00000100:00080000:1.0:1787826809.942432:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1787826809.942433:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787826809.942439:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826809.942445:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826809.942457:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626676736/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826809.942479:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826809/real 1787826809]  req@ffff8cd0d48f7840 x1874671626676352/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826809.942497:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626676352/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826809.942507:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826809.942510:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826809.942514:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626676352:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826809.942521:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626676480:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826809.942543:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826809.942555:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826809/real 1787826809]  req@ffff8cd0d48f5a40 x1874671626676480/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826809.942567:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626676480/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826809.942575:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826809.942578:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826809.942588:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626676480:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826809.942612:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626676608:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826809.942634:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826809.942645:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826809/real 1787826809]  req@ffff8cd0d48f43c0 x1874671626676608/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826809.942656:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626676608/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826809.942664:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826809.942667:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826809.942670:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626676608:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826809.942675:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626676736:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826809.942694:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826809.942705:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826809/real 1787826809]  req@ffff8cd0d48f70c0 x1874671626676736/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826809.942715:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626676736/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826809.942723:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826809.942726:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826809.942729:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626676736:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826810.134221:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826810.518190:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826811.158434:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826811.542254:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826812.182242:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826812.566246:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826813.206196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826813.590312:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826814.230211:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826814.614217:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787826815.062188:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826815.062197:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787826815.062202:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826815.062212:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 700
00000100:00080000:3.0:1787826815.062222:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826815.062229:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787826815.062264:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626676864/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826815.062288:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826815.062291:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826815.062293:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826815.062297:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 700
00000100:00080000:3.0:1787826815.062301:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826815.062304:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826815.062314:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826815.062324:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626676864:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826815.062328:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626676992/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826815.062340:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826815.062344:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787826815.062345:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826815.062349:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 700
00000100:00080000:3.0:1787826815.062352:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826815.062371:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826815.062376:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1787826815.062384:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826815.062388:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626677120/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826815.062399:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826815.062403:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826815.062404:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826815.062408:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 700
00000100:00080000:3.0:1787826815.062412:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826815.062416:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826815.062421:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826815.062423:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826815/real 1787826815]  req@ffff8cd0c8ead2c0 x1874671626676864/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826840 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826815.062432:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626677248/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826815.062444:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626676864/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826840 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826815.062455:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826815.062459:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826815.062463:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626676864:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826815.062490:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626676992:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826815.062511:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826815.062522:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826815/real 1787826815]  req@ffff8cd0c8eae1c0 x1874671626676992/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826840 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826815.062532:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626676992/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826840 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826815.062539:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826815.062542:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826815.062545:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626676992:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826815.062550:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626677120:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787826815.062565:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826815.062574:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826815/real 1787826815]  req@ffff8cd0c8eac780 x1874671626677120/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826840 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826815.062585:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626677120/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826840 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826815.062592:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826815.062595:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826815.062598:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626677120:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826815.062604:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626677248:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826815.062624:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826815.062634:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826815/real 1787826815]  req@ffff8cd0c8eac3c0 x1874671626677248/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826840 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826815.062650:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626677248/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826840 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826815.062658:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826815.062660:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826815.062663:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626677248:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826815.254640:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826815.638290:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826816.278199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826816.662223:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826817.302199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826817.686255:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826818.326204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826818.710171:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826819.350187:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826819.734246:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787826820.182242:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826820.182251:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787826820.182256:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826820.182265:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 706
00000100:00080000:2.0:1787826820.182275:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826820.182281:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787826820.182318:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8b40 x1874671626677376/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826820.182358:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826820.182361:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826820.182363:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826820.182367:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 706
00000100:00080000:2.0:1787826820.182379:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826820.182384:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826820.182393:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826820.182407:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626677504/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826820.182417:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826820.182419:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787826820.182421:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826820.182424:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 706
00000100:00080000:2.0:1787826820.182435:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826820.182438:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826820.182458:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826820.182469:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626677632/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826820.182478:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826820.182481:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826820.182483:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826820.182486:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 706
00000100:00080000:2.0:1787826820.182489:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826820.182493:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826820.182497:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826820.182506:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626677760/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826820.183235:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626677376:192.168.203.145@tcp:250:kworker.0
00000800:00000100:0.0:1787826820.183283:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826820.183314:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826820/real 1787826820]  req@ffff8cd0c3da8b40 x1874671626677376/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826845 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826820.183325:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8b40 x1874671626677376/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826845 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826820.183332:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826820.183334:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826820.183337:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626677376:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826820.183341:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626677504:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826820.183352:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826820.183359:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826820/real 1787826820]  req@ffff8cd0c3da8f00 x1874671626677504/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826845 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826820.183366:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626677504/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826845 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826820.183380:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826820.183383:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826820.183387:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626677504:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826820.183403:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626677632:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787826820.183428:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826820.183440:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826820/real 1787826820]  req@ffff8cd0c3da9680 x1874671626677632/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826845 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826820.183451:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626677632/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826845 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826820.183458:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826820.183461:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826820.183464:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626677632:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826820.183469:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626677760:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826820.183487:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826820.183494:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826820/real 1787826820]  req@ffff8cd0c3da83c0 x1874671626677760/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826845 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826820.183501:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626677760/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826845 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826820.183506:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826820.183508:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826820.183514:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626677760:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826820.374199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826820.758212:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826821.398164:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826821.782249:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826822.422189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826822.806272:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826823.446267:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826823.830324:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826824.470198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826824.854241:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787826825.303018:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826825.303025:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787826825.303029:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826825.303041:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 711
00000100:00080000:2.0:1787826825.303053:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826825.303060:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787826825.303112:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab480 x1874671626677888/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826825.303139:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826825.303142:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826825.303156:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826825.303161:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 711
00000100:00080000:2.0:1787826825.303165:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826825.303170:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826825.303179:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826825.303193:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626677888:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787826825.303197:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626678016/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826825.303210:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826825.303213:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787826825.303214:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826825.303218:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 711
00000100:00080000:2.0:1787826825.303222:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826825.303225:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826825.303231:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826825.303242:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626678144/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826825.303251:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826825.303254:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826825.303255:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826825.303261:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 711
00000800:00000100:0.0:1787826825.303264:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787826825.303266:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826825.303271:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826825.303276:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826825.303295:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626678272/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826825.303311:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826825/real 1787826825]  req@ffff8cd0c3dab480 x1874671626677888/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826850 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826825.303331:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab480 x1874671626677888/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826850 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826825.303341:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826825.303345:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826825.303349:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626677888:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826825.303369:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626678016:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826825.303390:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826825.303402:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826825/real 1787826825]  req@ffff8cd0c3daa940 x1874671626678016/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826850 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826825.303412:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626678016/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826850 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826825.303420:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826825.303423:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826825.303426:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626678016:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826825.303439:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626678144:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787826825.303459:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826825.303468:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826825/real 1787826825]  req@ffff8cd0c3dab0c0 x1874671626678144/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826850 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826825.303479:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626678144/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826850 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826825.303486:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826825.303489:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826825.303493:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626678144:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826825.303498:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626678272:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826825.303518:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826825.303528:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826825/real 1787826825]  req@ffff8cd0c3da9a40 x1874671626678272/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826850 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826825.303539:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626678272/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826850 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826825.303546:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826825.303549:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826825.303551:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626678272:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826825.494204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826825.878391:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826826.518242:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826826.902361:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826827.542190:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826827.926266:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826828.566201:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826828.950198:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826829.590387:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826829.974311:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787826830.422337:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826830.422346:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787826830.422350:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826830.422359:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 716
00000100:00080000:3.0:1787826830.422369:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826830.422376:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787826830.422411:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626678400/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826830.422437:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826830.422440:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826830.422442:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826830.422446:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 716
00000100:00080000:3.0:1787826830.422449:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826830.422452:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826830.422462:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826830.422474:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626678528/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826830.422485:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626678400:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787826830.422486:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826830.422492:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787826830.422494:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826830.422498:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 716
00000100:00080000:3.0:1787826830.422500:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826830.422503:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826830.422517:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826830.422531:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626678656/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826830.422540:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826830.422543:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826830.422545:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826830.422549:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 716
00000100:00080000:3.0:1787826830.422553:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826830.422557:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826830.422562:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1787826830.422571:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826830.422574:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626678784/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826830.422612:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826830/real 1787826830]  req@ffff8cd0c8eaf840 x1874671626678400/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826855 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826830.422640:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626678400/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826855 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826830.422647:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826830.422650:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826830.422654:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626678400:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826830.422678:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626678528:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826830.422700:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826830.422723:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826830/real 1787826830]  req@ffff8cd0c8eada40 x1874671626678528/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826855 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826830.422734:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626678528/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826855 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826830.422742:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826830.422745:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826830.422748:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626678528:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826830.422754:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626678656:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787826830.422775:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826830.422783:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826830/real 1787826830]  req@ffff8cd0c8eae940 x1874671626678656/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826855 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826830.422810:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626678656/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826855 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826830.422818:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826830.422820:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826830.422823:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626678656:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826830.422829:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626678784:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826830.422845:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826830.422854:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826830/real 1787826830]  req@ffff8cd0c8eaf0c0 x1874671626678784/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826855 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826830.422864:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626678784/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826855 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826830.422871:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826830.422874:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826830.422876:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626678784:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826830.614161:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826830.998218:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826831.638200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826832.022283:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826832.662227:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826833.046292:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826833.686195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826834.070282:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826834.710204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826835.094249:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787826835.542248:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826835.542261:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787826835.542266:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826835.542279:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 721
00000100:00080000:2.0:1787826835.542293:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826835.542302:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787826835.542348:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626678912/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826835.542383:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826835.542387:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826835.542389:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826835.542393:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 721
00000100:00080000:2.0:1787826835.542396:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826835.542400:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826835.542411:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826835.542422:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626679040/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826835.542431:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1787826835.542436:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626678912:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787826835.542456:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787826835.542458:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826835.542461:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 721
00000100:00080000:2.0:1787826835.542465:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826835.542468:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826835.542472:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826835.542481:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626679168/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826835.542488:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826835.542491:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826835.542492:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826835.542494:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 721
00000100:00080000:2.0:1787826835.542496:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826835.542498:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826835.542501:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826835.542507:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626679296/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787826835.542517:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826835.542572:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826835/real 1787826835]  req@ffff8cd0c3daa1c0 x1874671626678912/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826860 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826835.542603:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626678912/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826860 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826835.542628:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826835.542632:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826835.542636:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626678912:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826835.542662:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626679040:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826835.542681:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826835.542690:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826835/real 1787826835]  req@ffff8cd0c3dabc00 x1874671626679040/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826860 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826835.542698:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626679040/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826860 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826835.542703:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826835.542705:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826835.542707:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626679040:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826835.542711:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626679168:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787826835.542721:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826835.542726:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826835/real 1787826835]  req@ffff8cd0c3da8780 x1874671626679168/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826860 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826835.542733:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626679168/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826860 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826835.542738:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826835.542739:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826835.542741:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626679168:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826835.542749:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626679296:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826835.542762:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826835.542768:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826835/real 1787826835]  req@ffff8cd0c3dab840 x1874671626679296/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826860 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826835.542775:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626679296/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826860 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826835.542779:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826835.542781:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826835.542782:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626679296:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826835.734183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826836.118309:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826836.758169:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826837.142356:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826837.782171:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826838.166359:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826838.806155:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826839.190278:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826839.830146:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826840.214234:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787826840.662246:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826840.662255:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787826840.662260:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826840.662270:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 726
00000100:00080000:2.0:1787826840.662305:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826840.662312:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787826840.662345:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626679424/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826840.662375:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826840.662379:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826840.662381:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826840.662384:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 726
00000100:00080000:2.0:1787826840.662388:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826840.662392:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826840.662401:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826840.662413:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626679552/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826840.662424:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1787826840.662427:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626679424:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787826840.662429:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787826840.662432:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826840.662438:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 726
00000100:00080000:2.0:1787826840.662442:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826840.662446:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826840.662451:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826840.662463:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626679680/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826840.662472:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826840.662480:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826840.662482:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826840.662486:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 726
00000100:00080000:2.0:1787826840.662491:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826840.662497:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:0.0:1787826840.662498:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:2.0:1787826840.662504:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826840.662517:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626679808/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826840.662596:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826840/real 1787826840]  req@ffff8cd0c3da8000 x1874671626679424/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826865 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826840.662613:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626679424/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826865 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826840.662623:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826840.662627:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826840.662631:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626679424:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826840.662657:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626679552:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826840.662676:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826840.662685:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826840/real 1787826840]  req@ffff8cd0c3da9e00 x1874671626679552/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826865 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826840.662699:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626679552/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826865 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826840.662704:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826840.662706:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826840.662708:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626679552:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826840.662712:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626679680:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787826840.662725:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826840.662731:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826840/real 1787826840]  req@ffff8cd0c3da92c0 x1874671626679680/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826865 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826840.662738:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626679680/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826865 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826840.662742:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826840.662744:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826840.662746:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626679680:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826840.662749:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626679808:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826840.662757:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826840.662763:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826840/real 1787826840]  req@ffff8cd0c3daa580 x1874671626679808/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826865 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826840.662769:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626679808/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826865 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826840.662781:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826840.662783:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826840.662785:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626679808:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826840.854189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826841.238264:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826841.878273:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826842.262280:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826842.902407:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826843.286304:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826843.926181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826844.310407:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826844.950477:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826845.334252:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787826845.782387:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826845.782394:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787826845.782397:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826845.782405:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 731
00000100:00080000:3.0:1787826845.782413:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826845.782418:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787826845.782452:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626679936/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826845.782475:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826845.782478:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826845.782480:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826845.782484:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 731
00000100:00080000:3.0:1787826845.782487:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826845.782490:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826845.782499:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826845.782522:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626679936:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826845.782524:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626680064/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826845.782538:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826845.782542:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787826845.782544:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826845.782548:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 731
00000100:00080000:3.0:1787826845.782552:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826845.782556:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826845.782561:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1787826845.782574:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826845.782591:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626680192/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826845.782608:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1787826845.782612:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826845/real 1787826845]  req@ffff8cd0c8eaf0c0 x1874671626679936/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826870 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826845.782613:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826845.782616:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826845.782623:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 731
00000100:00080000:3.0:1787826845.782631:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826845.782636:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826845.782639:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626679936/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826870 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1787826845.782643:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1787826845.782656:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:3.0:1787826845.782658:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626680320/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826845.782663:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826845.782671:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626679936:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826845.782699:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626680064:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826845.782731:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826845.782744:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826845/real 1787826845]  req@ffff8cd0c8eae940 x1874671626680064/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826870 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826845.782755:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626680064/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826870 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826845.782762:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826845.782765:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826845.782767:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626680064:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826845.782780:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626680192:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787826845.782802:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826845.782813:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826845/real 1787826845]  req@ffff8cd0c8eada40 x1874671626680192/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826870 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826845.782824:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626680192/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826870 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826845.782832:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826845.782835:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826845.782838:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626680192:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826845.782844:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626680320:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826845.782861:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826845.782870:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826845/real 1787826845]  req@ffff8cd0c8eaf840 x1874671626680320/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826870 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826845.782881:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626680320/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826870 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826845.782888:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826845.782890:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826845.782893:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626680320:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826845.974194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826846.358222:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826846.998203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826847.382240:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826848.022194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826848.406237:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826849.046196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826849.430268:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826850.070200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826850.454275:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787826850.902196:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826850.902206:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787826850.902210:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826850.902219:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 736
00000100:00080000:3.0:1787826850.902229:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826850.902235:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787826850.902273:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626680448/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826850.902298:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826850.902313:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826850.902315:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826850.902319:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 736
00000100:00080000:3.0:1787826850.902323:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826850.902328:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826850.902337:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826850.902341:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626680448:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826850.902352:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626680576/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826850.902364:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826850.902368:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787826850.902370:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826850.902374:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 736
00000100:00080000:3.0:1787826850.902377:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826850.902382:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826850.902387:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826850.902399:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626680704/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826850.902406:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826850.902408:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826850.902410:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826850.902413:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 736
00000100:00080000:3.0:1787826850.902415:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826850.902418:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826850.902421:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626680576:192.168.203.145@tcp:38:kworker.0
00000008:00100000:3.0:1787826850.902429:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826850.902441:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626680832/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826850.902486:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626680704:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826850.902513:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626680832:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826851.094189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826851.478308:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826852.118203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826852.502266:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826853.143586:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826853.526332:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:1.0:1787826853.974375:0:2345:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:1.0:1787826853.974434:0:2345:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1787826853.974450:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787826853.974497:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787826853.974507:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787826853.974516:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:0.0:1787826853.974545:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826850/real 1787826853]  req@ffff8cd0c8ead2c0 x1874671626680832/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826875 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826853.974567:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626680832/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826875 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826853.974577:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826853.974582:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826853.974586:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626680832:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826853.974594:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826850/real 1787826853]  req@ffff8cd0c8eac780 x1874671626680576/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826875 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826853.974605:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626680576/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826875 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826853.974613:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826853.974616:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826853.974619:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626680576:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826853.974625:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826850/real 1787826853]  req@ffff8cd0c8eae1c0 x1874671626680704/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826875 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826853.974636:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626680704/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826875 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826853.974994:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826853.974998:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826853.975002:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626680704:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826853.975017:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826850/real 1787826853]  req@ffff8cd0c8eac3c0 x1874671626680448/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826875 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826853.975030:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626680448/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826875 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826853.975037:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826853.975041:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826853.975044:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626680448:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826854.166202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826854.550274:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826855.190240:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826855.574200:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787826856.022177:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826856.022184:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787826856.022188:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826856.022196:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 741
00000100:00080000:3.0:1787826856.022206:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826856.022211:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787826856.022237:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626680960/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826856.022260:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826856.022262:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826856.022263:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826856.022265:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 741
00000100:00080000:3.0:1787826856.022267:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826856.022269:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826856.022276:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826856.022286:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626681088/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826856.022308:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626680960:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787826856.022322:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826856.022325:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787826856.022327:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826856.022332:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 741
00000100:00080000:3.0:1787826856.022336:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826856.022340:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826856.022345:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826856.022358:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626681216/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826856.022367:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1787826856.022372:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787826856.022373:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826856.022377:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826856.022382:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 741
00000100:00080000:3.0:1787826856.022386:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826856.022391:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826856.022395:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826856.022407:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626681344/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826856.022415:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826856/real 1787826856]  req@ffff8cd0c8eacf00 x1874671626680960/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826881 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826856.022455:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626680960/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826881 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826856.022466:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826856.022470:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826856.022473:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626680960:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826856.022496:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626681088:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826856.022530:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826856.022547:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826856/real 1787826856]  req@ffff8cd0c8eac000 x1874671626681088/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826881 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826856.022559:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626681088/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826881 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826856.022566:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826856.022569:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826856.022572:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626681088:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826856.022577:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626681216:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787826856.022596:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826856.022606:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826856/real 1787826856]  req@ffff8cd0c8eac3c0 x1874671626681216/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826881 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826856.022617:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626681216/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826881 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826856.022625:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826856.022628:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826856.022631:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626681216:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826856.022637:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626681344:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826856.022655:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826856.022665:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826856/real 1787826856]  req@ffff8cd0c8eae1c0 x1874671626681344/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826881 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826856.022689:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626681344/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826881 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826856.022696:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826856.022699:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826856.022701:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626681344:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826856.214184:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826856.598277:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826857.238226:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826857.622261:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826858.262177:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826858.646265:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826859.286238:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826859.670289:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826860.310212:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826860.694328:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787826861.142188:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826861.142195:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787826861.142199:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826861.142205:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 746
00000100:00080000:3.0:1787826861.142215:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826861.142220:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787826861.142260:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626681472/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826861.142289:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826861.142292:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826861.142294:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826861.142297:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 746
00000100:00080000:3.0:1787826861.142300:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826861.142304:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826861.142313:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826861.142327:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626681600/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826861.142336:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826861.142339:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787826861.142341:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826861.142344:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 746
00000100:00080000:3.0:1787826861.142347:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826861.142350:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826861.142356:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826861.142370:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626681472:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826861.142384:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626681728/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826861.142397:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826861.142400:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826861.142413:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826861.142419:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 746
00000100:00080000:3.0:1787826861.142425:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826861.142430:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826861.142436:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826861.142449:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626681856/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787826861.142459:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826861.142512:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826861/real 1787826861]  req@ffff8cd0c8eac780 x1874671626681472/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826886 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826861.142526:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626681472/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826886 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826861.142535:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826861.142538:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826861.142541:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626681472:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826861.142547:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626681600:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826861.142563:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826861.142573:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826861/real 1787826861]  req@ffff8cd0c8ead2c0 x1874671626681600/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826886 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826861.142584:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626681600/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826886 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826861.142611:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826861.142614:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826861.142618:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626681600:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826861.142647:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626681728:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787826861.142672:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826861.142684:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826861/real 1787826861]  req@ffff8cd0c8eaf840 x1874671626681728/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826886 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826861.142694:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626681728/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826886 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826861.142699:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826861.142702:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826861.142703:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626681728:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826861.142707:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626681856:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826861.142721:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826861.142727:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826861/real 1787826861]  req@ffff8cd0c8eada40 x1874671626681856/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826886 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826861.142734:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626681856/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826886 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826861.142738:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826861.142740:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826861.142747:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626681856:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826861.334193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826861.718295:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826862.358511:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826862.742321:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826863.382208:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826863.766272:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826864.406193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826864.790465:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826865.430192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826865.814263:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787826866.262184:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826866.262192:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787826866.262200:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826866.262212:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 752
00000100:00080000:3.0:1787826866.262222:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826866.262227:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787826866.262258:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626681984/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826866.262277:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826866.262291:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826866.262292:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826866.262295:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 752
00000100:00080000:3.0:1787826866.262299:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826866.262302:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826866.262311:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826866.262316:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626681984:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826866.262321:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626682112/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826866.262330:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826866.262332:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787826866.262334:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826866.262336:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 752
00000100:00080000:3.0:1787826866.262338:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826866.262340:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826866.262343:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826866.262351:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626682240/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826866.262357:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826866.262359:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826866.262361:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826866.262362:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 752
00000100:00080000:3.0:1787826866.262365:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826866.262380:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:0.0:1787826866.262384:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:3.0:1787826866.262393:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826866.262406:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626682368/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826866.262417:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826866/real 1787826866]  req@ffff8cd0c8eae940 x1874671626681984/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826891 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826866.262441:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626681984/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826891 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826866.262449:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826866.262452:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826866.262457:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626681984:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826866.262475:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626682112:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826866.262490:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826866.262498:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826866/real 1787826866]  req@ffff8cd0c8eaf0c0 x1874671626682112/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826891 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826866.262505:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626682112/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826891 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826866.262510:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826866.262513:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826866.262516:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626682112:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826866.262522:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626682240:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787826866.262539:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826866.262559:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826866/real 1787826866]  req@ffff8cd0c8eada40 x1874671626682240/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826891 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826866.262570:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626682240/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826891 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826866.262577:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826866.262580:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826866.262584:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626682240:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826866.262590:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626682368:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826866.262608:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826866.262619:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826866/real 1787826866]  req@ffff8cd0c8eaf840 x1874671626682368/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826891 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826866.262630:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626682368/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826891 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826866.262637:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826866.262640:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826866.262643:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626682368:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826866.454315:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826866.838293:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826867.478228:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826867.862265:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826868.502192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826868.886347:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826869.526262:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826869.910299:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826870.550225:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826870.934328:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787826871.382325:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826871.382350:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787826871.382355:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826871.382367:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 757
00000100:00080000:2.0:1787826871.382382:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826871.382415:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787826871.382462:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daad00 x1874671626682496/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826871.382514:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826871.382518:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826871.382519:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826871.382524:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 757
00000100:00080000:2.0:1787826871.382529:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826871.382533:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826871.382556:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826871.382572:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626682624/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826871.382583:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826871.382587:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787826871.382588:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826871.382593:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 757
00000100:00080000:2.0:1787826871.382596:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826871.382600:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826871.382607:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826871.382618:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626682752/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826871.382627:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826871.382630:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826871.382637:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826871.382642:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 757
00000100:00080000:2.0:1787826871.382646:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826871.382651:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826871.382656:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826871.382667:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626682880/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826871.382755:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626682496:192.168.203.145@tcp:250:kworker.0
00000800:00000100:0.0:1787826871.382825:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826871.382881:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826871/real 1787826871]  req@ffff8cd0c3daad00 x1874671626682496/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826896 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826871.382898:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daad00 x1874671626682496/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826896 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826871.382920:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826871.382925:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826871.382930:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626682496:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826871.382937:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626682624:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826871.382960:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826871.382972:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826871/real 1787826871]  req@ffff8cd0c3daa580 x1874671626682624/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826896 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826871.382983:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626682624/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826896 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826871.383003:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826871.383006:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826871.383012:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626682624:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826871.383018:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626682752:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787826871.383036:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826871.383046:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826871/real 1787826871]  req@ffff8cd0c3da92c0 x1874671626682752/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826896 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826871.383055:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626682752/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826896 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826871.383062:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826871.383065:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826871.383067:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626682752:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826871.383078:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626682880:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826871.383148:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826871.383159:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826871/real 1787826871]  req@ffff8cd0c3da9e00 x1874671626682880/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826896 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826871.383169:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626682880/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826896 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826871.383176:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826871.383179:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826871.383182:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626682880:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826871.574247:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826871.959633:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826872.598219:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826872.982229:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826873.622219:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826874.006247:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826874.646229:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826875.030263:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826875.670178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826876.054249:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787826876.502216:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826876.502226:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787826876.502230:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826876.502239:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 762
00000100:00080000:1.0:1787826876.502250:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826876.502257:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787826876.502288:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626683008/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826876.502319:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826876.502321:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826876.502322:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826876.502325:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 762
00000100:00080000:1.0:1787826876.502328:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826876.502330:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826876.502338:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826876.502347:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626683136/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826876.502355:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826876.502357:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:0.0:1787826876.502366:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626683008:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787826876.502371:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826876.502376:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 762
00000100:00080000:1.0:1787826876.502379:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826876.502381:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826876.502387:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826876.502399:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626683264/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826876.502409:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826876.502413:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826876.502415:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826876.502419:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 762
00000100:00080000:1.0:1787826876.502423:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826876.502427:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:0.0:1787826876.502433:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:1.0:1787826876.502435:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826876.502450:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626683392/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826876.502476:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826876/real 1787826876]  req@ffff8cd0d48f61c0 x1874671626683008/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826901 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826876.502495:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626683008/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826901 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826876.502505:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826876.502509:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826876.502513:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626683008:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826876.502519:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626683136:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826876.502542:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826876.502553:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826876/real 1787826876]  req@ffff8cd0d48f7c00 x1874671626683136/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826901 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826876.502574:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626683136/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826901 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826876.502582:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826876.502585:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826876.502588:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626683136:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826876.502610:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626683264:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787826876.502631:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826876.502642:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826876/real 1787826876]  req@ffff8cd0d48f52c0 x1874671626683264/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826901 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826876.502653:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626683264/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826901 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826876.502661:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826876.502664:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826876.502667:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626683264:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826876.502672:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626683392:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826876.502688:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826876.502697:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826876/real 1787826876]  req@ffff8cd0d48f6d00 x1874671626683392/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826901 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826876.502706:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626683392/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826901 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826876.502723:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826876.502726:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826876.502729:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626683392:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826876.694201:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826877.078383:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826877.718158:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826878.102264:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826878.742208:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826879.126258:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826879.766465:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826880.150243:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826880.790211:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826881.174270:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787826881.622190:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826881.622207:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787826881.622212:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826881.622227:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 767
00000100:00080000:1.0:1787826881.622238:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826881.622246:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787826881.622310:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626683520/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826881.622338:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826881.622342:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826881.622343:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826881.622347:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 767
00000100:00080000:1.0:1787826881.622351:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826881.622354:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826881.622364:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826881.622375:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626683648/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826881.622382:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626683520:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787826881.622389:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826881.622393:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787826881.622395:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787826881.622400:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 767
00000100:00080000:1.0:1787826881.622403:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826881.622406:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826881.622411:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826881.622421:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626683776/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787826881.622430:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787826881.622433:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787826881.622434:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1787826881.622440:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787826881.622447:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 767
00000100:00080000:1.0:1787826881.622452:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787826881.622456:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787826881.622461:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787826881.622472:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626683904/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826881.622487:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826881/real 1787826881]  req@ffff8cd0d48f6d00 x1874671626683520/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826906 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826881.622504:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626683520/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826906 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826881.622515:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826881.622519:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826881.622523:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626683520:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826881.622545:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626683648:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826881.622563:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826881.622571:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826881/real 1787826881]  req@ffff8cd0d48f52c0 x1874671626683648/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826906 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826881.622581:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626683648/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826906 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826881.622589:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826881.622592:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826881.622595:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626683648:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826881.622607:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626683776:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787826881.622635:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826881.622646:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826881/real 1787826881]  req@ffff8cd0d48f7c00 x1874671626683776/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826906 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826881.622653:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626683776/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826906 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826881.622658:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826881.622660:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826881.622662:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626683776:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826881.622665:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626683904:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826881.622676:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826881.622682:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826881/real 1787826881]  req@ffff8cd0d48f61c0 x1874671626683904/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826906 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826881.622688:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626683904/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826906 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826881.622693:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826881.622694:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826881.622696:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626683904:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826881.814175:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826882.198270:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826882.838198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787826883.222246:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826883.862256:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826884.246260:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826884.886223:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826885.270231:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826885.910197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826886.294280:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787826886.742189:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826886.742196:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787826886.742199:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826886.742206:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 772
00000100:00080000:3.0:1787826886.742215:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826886.742219:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787826886.742270:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626684032/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826886.742290:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826886.742292:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826886.742293:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826886.742296:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 772
00000100:00080000:3.0:1787826886.742299:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826886.742301:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826886.742308:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826886.742315:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626684160/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826886.742325:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826886.742329:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787826886.742330:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826886.742334:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 772
00000100:00080000:3.0:1787826886.742338:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1787826886.742339:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626684032:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787826886.742345:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826886.742352:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826886.742364:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626684288/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826886.742375:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826886.742377:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826886.742379:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826886.742386:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 772
00000100:00080000:3.0:1787826886.742390:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826886.742393:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:0.0:1787826886.742400:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:3.0:1787826886.742405:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826886.742423:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626684416/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826886.742441:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826886/real 1787826886]  req@ffff8cd0c8ead2c0 x1874671626684032/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826911 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826886.742464:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626684032/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826911 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826886.742473:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826886.742477:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826886.742480:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626684032:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826886.742487:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626684160:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826886.742508:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826886.742518:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826886/real 1787826886]  req@ffff8cd0c8eac780 x1874671626684160/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826911 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826886.742528:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626684160/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826911 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826886.742535:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826886.742537:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826886.742540:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626684160:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826886.742562:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626684288:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787826886.742577:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826886.742586:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826886/real 1787826886]  req@ffff8cd0c8eae1c0 x1874671626684288/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826911 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826886.742595:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626684288/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826911 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826886.742611:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826886.742614:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826886.742617:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626684288:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826886.742622:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626684416:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826886.742639:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826886.742647:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826886/real 1787826886]  req@ffff8cd0c8eac3c0 x1874671626684416/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826911 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826886.742657:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626684416/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826911 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826886.742664:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826886.742666:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826886.742669:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626684416:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826886.934192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826887.318282:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826887.958197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826888.342264:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826888.982203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826889.367034:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826890.006251:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826890.390276:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826891.030191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826891.414254:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787826891.862171:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826891.862179:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787826891.862183:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826891.862206:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 777
00000100:00080000:3.0:1787826891.862216:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826891.862223:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787826891.862258:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626684544/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826891.862286:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826891.862290:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826891.862292:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826891.862295:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 777
00000100:00080000:3.0:1787826891.862299:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826891.862303:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826891.862324:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826891.862325:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626684544:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826891.862346:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626684672/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826891.862358:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826891.862362:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787826891.862363:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826891.862367:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 777
00000100:00080000:3.0:1787826891.862372:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1787826891.862377:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787826891.862378:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826891.862387:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826891.862413:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826891/real 1787826891]  req@ffff8cd0c8eac000 x1874671626684544/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826916 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826891.862415:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626684800/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826891.862439:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826891.862443:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:0.0:1787826891.862446:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626684544/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826916 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826891.862446:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826891.862456:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 777
00000100:00080000:3.0:1787826891.862463:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826891.862465:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826891.862470:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00080000:3.0:1787826891.862470:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826891.862476:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626684544:192.168.203.145@tcp:250:kworker.0
00000008:00100000:3.0:1787826891.862479:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826891.862492:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626684928/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826891.862509:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626684672:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826891.862535:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826891.862544:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826891/real 1787826891]  req@ffff8cd0c8eacf00 x1874671626684672/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826916 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826891.862552:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626684672/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826916 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826891.862558:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826891.862560:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826891.862562:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626684672:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826891.862565:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626684800:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787826891.862577:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826891.862583:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826891/real 1787826891]  req@ffff8cd0c8eac3c0 x1874671626684800/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826916 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826891.862594:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626684800/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826916 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826891.862601:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826891.862603:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826891.862606:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626684800:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826891.862612:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626684928:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826891.862630:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826891.862641:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826891/real 1787826891]  req@ffff8cd0c8eae1c0 x1874671626684928/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826916 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826891.862662:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626684928/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826916 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826891.862670:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826891.862673:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826891.862676:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626684928:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826892.054201:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826892.438270:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826893.078213:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826893.462318:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826894.102274:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826894.486309:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826895.126191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826895.510271:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826896.150258:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826896.534348:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787826896.982202:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826896.982215:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787826896.982219:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826896.982226:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 782
00000100:00080000:3.0:1787826896.982239:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826896.982244:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787826896.982272:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626685056/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826896.982296:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826896.982298:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826896.982300:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826896.982302:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 782
00000100:00080000:3.0:1787826896.982304:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826896.982306:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826896.982314:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826896.982322:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626685184/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826896.982330:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826896.982332:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787826896.982333:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826896.982336:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 782
00000100:00080000:3.0:1787826896.982337:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826896.982354:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826896.982359:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826896.982372:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626685312/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826896.982384:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626685056:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787826896.982388:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826896.982395:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826896.982398:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826896.982403:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 782
00000100:00080000:3.0:1787826896.982407:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826896.982412:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826896.982417:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826896.982439:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626685440/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787826896.982462:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826896.982510:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826896/real 1787826896]  req@ffff8cd0c8eac780 x1874671626685056/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826921 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826896.982530:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626685056/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826921 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826896.982544:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826896.982548:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826896.982560:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626685056:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826896.982569:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626685184:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826896.982593:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826896.982606:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826896/real 1787826896]  req@ffff8cd0c8ead2c0 x1874671626685184/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826921 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826896.982619:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626685184/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826921 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826896.982645:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826896.982649:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826896.982653:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626685184:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826896.982694:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626685312:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787826896.982720:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826896.982733:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826896/real 1787826896]  req@ffff8cd0c8eaf840 x1874671626685312/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826921 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826896.982743:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626685312/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826921 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826896.982751:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826896.982754:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826896.982757:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626685312:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826896.982764:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626685440:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826896.982784:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826896.982795:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826896/real 1787826896]  req@ffff8cd0c8eada40 x1874671626685440/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826921 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826896.982806:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626685440/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826921 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826896.982814:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826896.982817:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826896.982827:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626685440:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826897.174187:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826897.558814:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826898.198234:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826898.582284:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826899.222215:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826899.606274:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826900.246199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826900.630272:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826901.270216:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826901.654690:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787826902.102193:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826902.102200:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787826902.102204:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826902.102211:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 787
00000100:00080000:3.0:1787826902.102219:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826902.102225:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787826902.102255:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626685568/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826902.102287:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826902.102290:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826902.102291:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826902.102294:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 787
00000100:00080000:3.0:1787826902.102296:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826902.102299:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826902.102306:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826902.102315:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626685696/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826902.102324:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1787826902.102329:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626685568:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787826902.102330:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787826902.102335:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826902.102341:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 787
00000100:00080000:3.0:1787826902.102345:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826902.102349:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826902.102354:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826902.102368:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626685824/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826902.102377:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826902.102380:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826902.102382:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1787826902.102388:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787826902.102388:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 787
00000100:00080000:3.0:1787826902.102400:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826902.102406:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826902.102412:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826902.102424:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626685952/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826902.102446:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826902/real 1787826902]  req@ffff8cd0c8eaf0c0 x1874671626685568/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826927 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826902.102464:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626685568/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826927 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826902.102471:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826902.102474:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826902.102480:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626685568:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826902.102501:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626685696:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826902.102517:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826902.102526:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826902/real 1787826902]  req@ffff8cd0c8eae940 x1874671626685696/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826927 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826902.102533:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626685696/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826927 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826902.102538:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826902.102540:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826902.102542:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626685696:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826902.102545:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626685824:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787826902.102555:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826902.102568:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826902/real 1787826902]  req@ffff8cd0c8eada40 x1874671626685824/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826927 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826902.102579:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626685824/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826927 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826902.102586:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826902.102589:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826902.102592:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626685824:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826902.102598:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626685952:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826902.102616:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826902.102635:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826902/real 1787826902]  req@ffff8cd0c8eaf840 x1874671626685952/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826927 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826902.102650:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626685952/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826927 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826902.102658:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826902.102661:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826902.102664:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626685952:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826902.294159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826902.678216:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826903.318180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826903.702257:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826904.342160:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826904.726259:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826905.366168:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826905.750329:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826906.390166:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826906.774305:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787826907.222241:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826907.222250:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787826907.222254:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826907.222262:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 793
00000100:00080000:2.0:1787826907.222273:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826907.222279:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787826907.222311:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626686080/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826907.222334:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826907.222337:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826907.222339:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826907.222342:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 793
00000100:00080000:2.0:1787826907.222345:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826907.222362:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826907.222371:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826907.222381:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626686080:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787826907.222385:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626686208/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826907.222398:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826907.222402:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787826907.222404:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826907.222408:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 793
00000100:00080000:2.0:1787826907.222413:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826907.222417:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826907.222422:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826907.222432:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626686336/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826907.222441:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1787826907.222445:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787826907.222447:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826907.222450:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826907.222455:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 793
00000100:00080000:2.0:1787826907.222459:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826907.222463:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826907.222467:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826907.222477:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626686464/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826907.222488:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826907/real 1787826907]  req@ffff8cd0c3da8000 x1874671626686080/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826932 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826907.222501:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626686080/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826932 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826907.222515:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826907.222518:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826907.222520:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626686080:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826907.222538:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626686208:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826907.222560:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826907.222573:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826907/real 1787826907]  req@ffff8cd0c3dab840 x1874671626686208/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826932 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826907.222584:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626686208/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826932 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826907.222592:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826907.222595:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826907.222598:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626686208:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826907.222602:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626686336:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787826907.222616:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826907.222626:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826907/real 1787826907]  req@ffff8cd0c3da8780 x1874671626686336/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826932 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826907.222637:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626686336/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826932 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826907.222644:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826907.222647:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826907.222650:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626686336:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826907.222662:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626686464:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826907.222681:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826907.222691:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826907/real 1787826907]  req@ffff8cd0c3dabc00 x1874671626686464/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826932 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826907.222702:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626686464/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826932 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826907.222710:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826907.222713:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826907.222716:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626686464:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826907.414138:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826907.798266:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826908.438182:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826908.822235:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826909.462197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826909.846250:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826910.486280:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826910.870227:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826911.510188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826911.894303:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787826912.342197:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826912.342222:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787826912.342226:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826912.342236:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 798
00000100:00080000:2.0:1787826912.342245:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826912.342252:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787826912.342282:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626686592/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826912.342313:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826912.342317:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826912.342319:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826912.342322:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 798
00000100:00080000:2.0:1787826912.342326:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826912.342330:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826912.342339:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826912.342352:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626686720/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826912.342362:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626686592:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787826912.342364:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826912.342371:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787826912.342373:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826912.342377:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 798
00000100:00080000:2.0:1787826912.342381:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826912.342385:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826912.342390:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826912.342400:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626686848/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826912.342415:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826912.342418:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826912.342420:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826912.342424:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 798
00000100:00080000:2.0:1787826912.342429:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1787826912.342434:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787826912.342436:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826912.342443:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826912.342456:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626686976/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826912.342485:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826912/real 1787826912]  req@ffff8cd0c3daa1c0 x1874671626686592/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826937 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826912.342504:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626686592/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826937 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826912.342513:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826912.342517:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826912.342525:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626686592:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826912.342548:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626686720:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826912.342571:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826912.342585:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826912/real 1787826912]  req@ffff8cd0c3da9a40 x1874671626686720/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826937 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826912.342606:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626686720/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826937 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826912.342614:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826912.342618:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826912.342621:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626686720:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826912.342626:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626686848:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787826912.342648:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826912.342659:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826912/real 1787826912]  req@ffff8cd0c3dab0c0 x1874671626686848/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826937 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826912.342670:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626686848/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826937 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826912.342677:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826912.342680:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826912.342683:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626686848:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826912.342688:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626686976:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826912.342723:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826912.342734:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826912/real 1787826912]  req@ffff8cd0c3daa940 x1874671626686976/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826937 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826912.342744:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626686976/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826937 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826912.342763:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826912.342766:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826912.342770:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626686976:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826912.534236:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826912.918250:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826913.558217:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826913.942237:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826914.582166:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826914.966270:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826915.606198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826915.990235:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826916.630187:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826917.014320:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787826917.462199:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826917.462208:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787826917.462212:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826917.462221:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 803
00000100:00080000:2.0:1787826917.462232:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826917.462239:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787826917.462285:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab480 x1874671626687104/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826917.462308:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826917.462310:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826917.462311:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826917.462314:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 803
00000100:00080000:2.0:1787826917.462318:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826917.462322:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826917.462344:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826917.462358:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626687232/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826917.462362:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626687104:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787826917.462372:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826917.462375:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787826917.462378:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826917.462384:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 803
00000100:00080000:2.0:1787826917.462388:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826917.462392:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826917.462398:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826917.462411:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626687360/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826917.462420:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826917.462423:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826917.462425:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826917.462429:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 803
00000100:00080000:2.0:1787826917.462432:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826917.462449:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826917.462455:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826917.462466:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626687488/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826917.462499:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626687232:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826917.462533:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626687360:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826917.462571:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626687488:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826917.654200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826918.038292:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826918.678259:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826919.062253:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826919.702189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826920.086224:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:3.0:1787826920.534348:0:2344:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:3.0:1787826920.534388:0:2344:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:3.0:1787826920.534403:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:3.0:1787826920.534443:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:3.0:1787826920.534452:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:3.0:1787826920.534459:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:0.0:1787826920.534558:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826917/real 1787826920]  req@ffff8cd0c3da83c0 x1874671626687232/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826942 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826920.534581:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626687232/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826942 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826920.534592:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826920.534604:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826920.534609:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626687232:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826920.534616:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826917/real 1787826920]  req@ffff8cd0c3da9680 x1874671626687360/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826942 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826920.534628:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626687360/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826942 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826920.534636:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826920.534645:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826920.534649:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626687360:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826920.534656:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826917/real 1787826920]  req@ffff8cd0c3da8f00 x1874671626687488/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826942 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826920.534674:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626687488/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826942 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826920.534682:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826920.534685:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826920.534689:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626687488:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826920.534694:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826917/real 1787826920]  req@ffff8cd0c3dab480 x1874671626687104/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826942 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826920.534706:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab480 x1874671626687104/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826942 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826920.534713:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826920.534716:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826920.534719:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626687104:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826920.726213:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826921.110252:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826921.750204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826922.134297:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787826922.582245:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826922.582254:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787826922.582258:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826922.582266:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 808
00000100:00080000:3.0:1787826922.582275:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826922.582281:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787826922.582312:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626687616/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826922.582339:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826922.582342:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826922.582344:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826922.582348:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 808
00000100:00080000:3.0:1787826922.582351:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826922.582355:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826922.582365:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826922.582376:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626687616:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826922.582378:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626687744/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826922.582392:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826922.582396:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787826922.582397:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826922.582401:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 808
00000100:00080000:3.0:1787826922.582404:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826922.582421:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826922.582426:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826922.582437:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626687872/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787826922.582446:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787826922.582451:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826922.582455:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826922.582457:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826922.582461:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 808
00000100:00080000:3.0:1787826922.582466:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826922.582470:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826922.582475:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826922.582486:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626688000/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826922.582490:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826922/real 1787826922]  req@ffff8cd0c8ead2c0 x1874671626687616/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826947 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826922.582513:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626687616/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826947 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826922.582531:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826922.582536:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826922.582539:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626687616:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826922.582559:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626687744:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826922.582576:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826922.582586:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826922/real 1787826922]  req@ffff8cd0c8eac780 x1874671626687744/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826947 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826922.582597:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626687744/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826947 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826922.582604:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826922.582606:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826922.582609:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626687744:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826922.582622:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626687872:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787826922.582641:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826922.582652:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826922/real 1787826922]  req@ffff8cd0c8eae1c0 x1874671626687872/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826947 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826922.582663:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626687872/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826947 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826922.582669:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826922.582672:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826922.582675:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626687872:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826922.582688:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626688000:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826922.582717:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826922.582728:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826922/real 1787826922]  req@ffff8cd0c8eac3c0 x1874671626688000/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826947 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826922.582738:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626688000/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826947 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826922.582745:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826922.582747:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826922.582751:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626688000:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826922.774235:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826923.158244:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826923.798179:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826924.182275:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826924.822199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826925.206764:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826925.846170:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826926.230244:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826926.870187:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826927.254283:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787826927.702244:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826927.702253:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787826927.702276:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826927.702286:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 813
00000100:00080000:2.0:1787826927.702298:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826927.702305:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787826927.702341:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8b40 x1874671626688128/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826927.702374:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826927.702378:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826927.702380:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826927.702384:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 813
00000100:00080000:2.0:1787826927.702388:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826927.702393:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826927.702405:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826927.702417:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab480 x1874671626688256/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826927.702424:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626688128:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787826927.702429:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826927.702434:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787826927.702436:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826927.702440:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 813
00000100:00080000:2.0:1787826927.702444:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826927.702448:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826927.702455:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826927.702468:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626688384/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826927.702486:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826927.702488:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826927.702490:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826927.702493:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 813
00000100:00080000:2.0:1787826927.702497:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1787826927.702500:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787826927.702504:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826927.702509:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826927.702519:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626688512/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826927.702555:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826927/real 1787826927]  req@ffff8cd0c3da8b40 x1874671626688128/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826952 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826927.702575:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8b40 x1874671626688128/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826952 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826927.702586:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826927.702590:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826927.702594:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626688128:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826927.702622:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626688256:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826927.702643:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826927.702653:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826927/real 1787826927]  req@ffff8cd0c3dab480 x1874671626688256/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826952 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826927.702667:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab480 x1874671626688256/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826952 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826927.702672:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826927.702674:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826927.702677:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626688256:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826927.702681:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626688384:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787826927.702695:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826927.702701:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826927/real 1787826927]  req@ffff8cd0c3da8f00 x1874671626688384/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826952 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826927.702708:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626688384/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826952 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826927.702713:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826927.702715:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826927.702717:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626688384:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826927.702722:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626688512:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826927.702738:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826927.702747:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826927/real 1787826927]  req@ffff8cd0c3da9680 x1874671626688512/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826952 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826927.702759:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626688512/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826952 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826927.702777:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826927.702779:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826927.702781:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626688512:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826927.894198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826928.278234:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826928.918196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826929.302244:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826929.942187:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826930.326252:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826930.966946:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826931.350251:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826931.990195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826932.374261:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787826932.822211:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826932.822221:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787826932.822225:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826932.822243:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 818
00000100:00080000:2.0:1787826932.822254:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826932.822260:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787826932.822297:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626688640/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826932.822325:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826932.822328:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826932.822336:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826932.822340:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 818
00000100:00080000:2.0:1787826932.822344:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826932.822349:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826932.822365:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626688640:192.168.203.145@tcp:250:kworker.0
00000008:00100000:2.0:1787826932.822373:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826932.822399:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626688768/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826932.822410:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826932.822413:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787826932.822414:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826932.822417:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 818
00000100:00080000:2.0:1787826932.822422:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1787826932.822426:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787826932.822430:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826932.822435:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826932.822446:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626688896/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826932.822453:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826932.822456:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826932.822458:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826932.822461:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 818
00000100:00080000:2.0:1787826932.822465:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826932.822469:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826932.822472:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826932/real 1787826932]  req@ffff8cd0c3da83c0 x1874671626688640/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826957 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1787826932.822475:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826932.822485:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626688640/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826957 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826932.822495:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:2.0:1787826932.822498:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626689024/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826932.822500:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826932.822506:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626688640:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826932.822544:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626688768:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826932.822566:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826932.822578:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826932/real 1787826932]  req@ffff8cd0c3daa940 x1874671626688768/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826957 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826932.822588:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626688768/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826957 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826932.822593:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826932.822595:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826932.822597:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626688768:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826932.822611:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626688896:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787826932.822632:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826932.822643:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826932/real 1787826932]  req@ffff8cd0c3dab0c0 x1874671626688896/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826957 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826932.822654:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626688896/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826957 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826932.822662:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826932.822665:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826932.822667:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626688896:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826932.822671:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626689024:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826932.822686:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826932.822693:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826932/real 1787826932]  req@ffff8cd0c3da9a40 x1874671626689024/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826957 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826932.822699:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626689024/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826957 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826932.822704:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826932.822705:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826932.822707:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626689024:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826933.014182:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826933.398228:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826934.038180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826934.422256:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826935.062163:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826935.446240:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826936.086187:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826936.470268:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826937.110185:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826937.494299:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787826937.942257:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826937.942267:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787826937.942272:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826937.942299:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 823
00000100:00080000:0.0:1787826937.942308:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826937.942315:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826937.942351:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d2c0 x1874671626689152/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826937.942379:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826937.942382:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826937.942384:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826937.942388:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 823
00000100:00080000:0.0:1787826937.942391:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826937.942395:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826937.942403:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826937.942416:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626689280/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826937.942427:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826937.942432:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:3.0:1787826937.942432:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626689152:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787826937.942435:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826937.942444:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 823
00000100:00080000:0.0:1787826937.942447:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826937.942451:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826937.942455:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826937.942465:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626689408/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826937.942484:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826937.942487:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826937.942488:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826937.942492:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 823
00000100:00080000:0.0:1787826937.942497:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826937.942502:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:3.0:1787826937.942505:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:0.0:1787826937.942507:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826937.942531:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626689536/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826937.942548:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826937/real 1787826937]  req@ffff8cd0c8e9d2c0 x1874671626689152/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826962 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826937.942566:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d2c0 x1874671626689152/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826962 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826937.942576:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826937.942580:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826937.942589:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626689152:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826937.942610:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626689280:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826937.942632:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826937.942644:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826937/real 1787826937]  req@ffff8cd0c8e9c3c0 x1874671626689280/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826962 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826937.942654:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626689280/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826962 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826937.942669:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826937.942672:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826937.942674:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626689280:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826937.942680:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626689408:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826937.942696:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826937.942705:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826937/real 1787826937]  req@ffff8cd0c8e9cb40 x1874671626689408/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826962 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826937.942722:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626689408/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826962 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826937.942729:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826937.942731:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826937.942734:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626689408:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826937.942739:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626689536:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826937.942755:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826937.942764:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826937/real 1787826937]  req@ffff8cd0c8e9f480 x1874671626689536/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826962 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826937.942773:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626689536/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826962 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826937.942779:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826937.942781:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826937.942784:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626689536:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826938.134207:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826938.518275:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826939.158201:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826939.542620:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826940.182215:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826940.566346:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826941.206162:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826941.590207:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826942.230206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826942.614290:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787826943.062190:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826943.062201:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787826943.062205:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826943.062215:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 828
00000100:00080000:0.0:1787826943.062225:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826943.062232:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826943.062265:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626689664/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826943.062288:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826943.062290:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826943.062292:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826943.062294:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 828
00000100:00080000:0.0:1787826943.062297:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826943.062300:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826943.062321:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826943.062323:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626689664:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826943.062337:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626689792/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826943.062350:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826943.062354:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787826943.062356:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826943.062359:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 828
00000100:00080000:0.0:1787826943.062364:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826943.062367:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826943.062372:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826943.062385:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626689920/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787826943.062387:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787826943.062397:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826943.062400:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826943.062402:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826943.062407:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 828
00000100:00080000:0.0:1787826943.062410:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826943.062414:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826943.062419:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826943.062434:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826943/real 1787826943]  req@ffff8cd0c8e9c000 x1874671626689664/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826968 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826943.062435:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626690048/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826943.062472:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626689664/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826968 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826943.062483:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826943.062487:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826943.062491:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626689664:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826943.062514:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626689792:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826943.062538:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826943.062550:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826943/real 1787826943]  req@ffff8cd0c8e9f840 x1874671626689792/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826968 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826943.062563:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626689792/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826968 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826943.062571:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826943.062574:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826943.062578:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626689792:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826943.062583:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626689920:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826943.062605:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826943.062616:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826943/real 1787826943]  req@ffff8cd0c8e9ed00 x1874671626689920/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826968 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826943.062628:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626689920/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826968 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826943.062645:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826943.062649:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826943.062651:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626689920:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826943.062670:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626690048:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826943.062701:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826943.062711:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826943/real 1787826943]  req@ffff8cd0c8e9d680 x1874671626690048/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826968 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826943.062722:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626690048/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826968 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826943.062729:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826943.062732:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826943.062735:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626690048:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826943.254397:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826943.638285:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826944.278207:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826944.662311:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826945.302203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826945.686307:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826946.326226:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826946.710439:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826947.350191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826947.734303:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787826948.182199:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826948.182215:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787826948.182233:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826948.182241:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 834
00000100:00080000:0.0:1787826948.182251:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826948.182258:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826948.182288:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626690176/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826948.182315:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826948.182319:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826948.182320:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826948.182324:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 834
00000100:00080000:0.0:1787826948.182327:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826948.182331:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826948.182340:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826948.182352:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626690304/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826948.182359:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626690176:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787826948.182365:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826948.182370:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787826948.182372:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826948.182376:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 834
00000100:00080000:0.0:1787826948.182380:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826948.182384:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826948.182389:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826948.182400:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626690432/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787826948.182420:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787826948.182422:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826948.182427:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826948.182429:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826948.182434:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 834
00000100:00080000:0.0:1787826948.182439:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826948.182444:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826948.182449:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826948.182461:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626690560/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826948.182467:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826948/real 1787826948]  req@ffff8cd0c8e9cf00 x1874671626690176/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826973 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826948.182490:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626690176/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826973 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826948.182503:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826948.182506:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826948.182509:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626690176:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826948.182526:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626690304:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826948.182543:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826948.182555:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826948/real 1787826948]  req@ffff8cd0c8e9e940 x1874671626690304/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826973 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826948.182566:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626690304/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826973 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826948.182573:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826948.182576:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826948.182586:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626690304:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826948.182596:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626690432:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826948.182615:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826948.182623:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826948/real 1787826948]  req@ffff8cd0c8e9e580 x1874671626690432/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826973 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826948.182630:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626690432/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826973 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826948.182635:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826948.182637:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826948.182639:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626690432:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826948.182647:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626690560:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826948.182659:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826948.182665:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826948/real 1787826948]  req@ffff8cd0c8e9de00 x1874671626690560/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826973 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826948.182672:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626690560/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826973 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826948.182676:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826948.182678:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826948.182679:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626690560:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826948.374215:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826948.758314:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826949.398193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826949.782312:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826950.422184:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826950.806689:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826951.446203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826951.830819:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826952.470246:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826952.854264:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787826953.302219:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826953.302226:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787826953.302243:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826953.302251:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 839
00000100:00080000:2.0:1787826953.302259:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826953.302264:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787826953.302293:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626690688/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826953.302319:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826953.302323:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826953.302325:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826953.302328:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 839
00000100:00080000:2.0:1787826953.302335:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826953.302338:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826953.302348:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826953.302358:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626690816/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826953.302365:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826953.302368:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787826953.302369:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826953.302371:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 839
00000100:00080000:2.0:1787826953.302373:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826953.302378:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787826953.302378:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626690688:192.168.203.145@tcp:250:kworker.0
00000008:00100000:2.0:1787826953.302382:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826953.302391:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626690944/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826953.302419:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826953.302422:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826953.302424:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826953.302427:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 839
00000100:00080000:2.0:1787826953.302432:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826953.302436:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826953.302441:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826953.302452:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626691072/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787826953.302545:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826953.302583:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826953/real 1787826953]  req@ffff8cd0c3daa1c0 x1874671626690688/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826978 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826953.302599:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626690688/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826978 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826953.302610:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826953.302614:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826953.302617:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626690688:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826953.302631:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626690816:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826953.302654:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826953.302665:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826953/real 1787826953]  req@ffff8cd0c3dabc00 x1874671626690816/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826978 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826953.302677:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626690816/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826978 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826953.302684:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826953.302687:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826953.302690:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626690816:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826953.302711:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626690944:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826953.302729:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826953.302739:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826953/real 1787826953]  req@ffff8cd0c3da8780 x1874671626690944/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826978 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826953.302750:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626690944/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826978 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826953.302757:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826953.302760:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826953.302775:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626690944:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826953.302781:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626691072:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826953.302800:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826953.302810:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826953/real 1787826953]  req@ffff8cd0c3dab840 x1874671626691072/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826978 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826953.302826:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626691072/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826978 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826953.302833:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826953.302836:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826953.302838:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626691072:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826953.494170:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826953.878226:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826954.518201:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826954.902785:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826955.542211:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826955.926354:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826956.566205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826956.950549:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826957.590211:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826957.974781:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787826958.422244:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826958.422260:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787826958.422265:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826958.422275:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 844
00000100:00080000:3.0:1787826958.422286:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826958.422293:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787826958.422330:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626691200/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826958.422373:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826958.422376:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826958.422379:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826958.422387:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 844
00000100:00080000:3.0:1787826958.422392:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826958.422416:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826958.422427:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826958.422442:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626691328/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826958.422455:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826958.422463:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:0.0:1787826958.422466:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626691200:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787826958.422467:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826958.422481:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 844
00000100:00080000:3.0:1787826958.422487:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826958.422491:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826958.422508:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826958.422600:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626691456/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826958.422610:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826958.422613:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826958.422615:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826958.422619:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 844
00000100:00080000:3.0:1787826958.422623:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826958.422628:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826958.422632:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826958.422641:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626691584/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787826958.422828:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826958.422886:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826958/real 1787826958]  req@ffff8cd0c8eacf00 x1874671626691200/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826983 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826958.422962:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626691200/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826983 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826958.422976:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826958.422981:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826958.422986:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626691200:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826958.423017:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626691328:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826958.423051:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826958.423062:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826958/real 1787826958]  req@ffff8cd0c8eac000 x1874671626691328/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826983 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826958.423124:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626691328/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826983 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826958.423134:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826958.423137:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826958.423140:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626691328:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826958.423147:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626691456:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787826958.423254:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826958.423286:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826958/real 1787826958]  req@ffff8cd0c8eac3c0 x1874671626691456/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826983 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826958.423304:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626691456/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826983 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826958.423315:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826958.423319:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826958.423324:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626691456:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826958.423334:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626691584:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826958.423357:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826958.423369:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826958/real 1787826958]  req@ffff8cd0c8eae1c0 x1874671626691584/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826983 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826958.423382:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626691584/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826983 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826958.423473:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826958.423478:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826958.423482:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626691584:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787826958.614204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826958.998282:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826959.638200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826960.022294:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826960.662225:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826961.046418:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787826961.686202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826962.070307:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787826962.710857:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826963.096544:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787826963.542210:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826963.542220:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787826963.542224:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826963.542234:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 849
00000100:00080000:3.0:1787826963.542245:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826963.542253:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787826963.542290:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626691712/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826963.542320:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826963.542323:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826963.542325:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826963.542329:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 849
00000100:00080000:3.0:1787826963.542333:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826963.542337:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826963.542346:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826963.542358:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626691840/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826963.542368:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1787826963.542369:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626691712:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787826963.542373:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787826963.542378:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826963.542382:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 849
00000100:00080000:3.0:1787826963.542385:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826963.542402:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826963.542408:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826963.542419:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626691968/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826963.542428:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826963.542432:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826963.542433:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826963.542437:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 849
00000800:00000100:0.0:1787826963.542441:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787826963.542443:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826963.542449:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826963.542455:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826963.542465:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626692096/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826963.542484:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826963/real 1787826963]  req@ffff8cd0c8eac780 x1874671626691712/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826988 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826963.542508:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626691712/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826988 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826963.542518:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826963.542522:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826963.542526:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626691712:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826963.542548:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626691840:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826963.542569:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826963.542581:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826963/real 1787826963]  req@ffff8cd0c8ead2c0 x1874671626691840/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826988 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826963.542592:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626691840/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826988 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826963.542600:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826963.542603:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826963.542606:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626691840:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826963.542620:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626691968:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787826963.542639:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826963.542650:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826963/real 1787826963]  req@ffff8cd0c8eaf840 x1874671626691968/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826988 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826963.542669:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626691968/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826988 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826963.542677:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826963.542680:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826963.542683:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626691968:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826963.542689:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626692096:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826963.542717:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826963.542727:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826963/real 1787826963]  req@ffff8cd0c8eada40 x1874671626692096/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826988 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826963.542738:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626692096/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826988 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826963.542759:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826963.542763:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826963.542766:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626692096:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787826963.734205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826964.118302:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787826964.758239:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826965.142248:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787826965.782178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826966.166279:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787826966.806203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826967.190284:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787826967.830213:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826968.214271:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787826968.662287:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826968.662297:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787826968.662300:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826968.662310:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 854
00000100:00080000:3.0:1787826968.662320:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826968.662327:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787826968.662362:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626692224/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826968.662393:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826968.662397:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826968.662411:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826968.662420:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 854
00000100:00080000:3.0:1787826968.662425:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826968.662429:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826968.662439:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826968.662454:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626692352/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826968.662465:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826968.662468:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787826968.662470:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826968.662474:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 854
00000100:00100000:0.0:1787826968.662478:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626692224:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787826968.662479:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826968.662484:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826968.662489:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826968.662502:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626692480/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826968.662513:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826968.662517:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826968.662519:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826968.662523:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 854
00000100:00080000:3.0:1787826968.662527:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826968.662531:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826968.662535:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1787826968.662547:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826968.662555:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626692608/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826968.662590:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826968/real 1787826968]  req@ffff8cd0c8eae940 x1874671626692224/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826993 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826968.662616:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626692224/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826993 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826968.662625:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826968.662633:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826968.662637:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626692224:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826968.662644:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626692352:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826968.662679:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826968.662691:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826968/real 1787826968]  req@ffff8cd0c8eaf0c0 x1874671626692352/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826993 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826968.662703:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626692352/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826993 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826968.662711:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826968.662714:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826968.662717:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626692352:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826968.662740:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626692480:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787826968.662758:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826968.662767:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826968/real 1787826968]  req@ffff8cd0c8eada40 x1874671626692480/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826993 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826968.662778:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626692480/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826993 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826968.662796:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826968.662799:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826968.662802:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626692480:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826968.662808:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626692608:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826968.662827:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826968.662847:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826968/real 1787826968]  req@ffff8cd0c8eaf840 x1874671626692608/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826993 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826968.662858:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626692608/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826993 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826968.662865:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826968.662867:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826968.662870:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626692608:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787826968.854217:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826969.238267:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787826969.878223:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826970.262837:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787826970.902237:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826971.286261:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787826971.926200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826972.310327:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787826972.950148:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826973.334443:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787826973.782203:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826973.782212:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787826973.782216:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826973.782224:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 859
00000100:00080000:2.0:1787826973.782233:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826973.782237:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787826973.782266:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626692736/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826973.782297:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826973.782300:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826973.782302:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826973.782306:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 859
00000100:00080000:2.0:1787826973.782310:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826973.782314:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826973.782334:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826973.782339:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626692736:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787826973.782346:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626692864/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826973.782355:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826973.782361:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787826973.782363:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826973.782365:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 859
00000100:00080000:2.0:1787826973.782369:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826973.782374:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826973.782379:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826973.782390:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626692992/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787826973.782400:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787826973.782400:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826973.782406:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826973.782408:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826973.782413:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 859
00000100:00080000:2.0:1787826973.782417:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826973.782420:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826973.782425:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826973.782433:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626693120/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826973.782440:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826973/real 1787826973]  req@ffff8cd0c3da8000 x1874671626692736/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826998 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826973.782453:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626692736/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826998 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826973.782460:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826973.782464:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826973.782467:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626692736:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826973.782494:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626692864:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826973.782529:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826973.782539:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826973/real 1787826973]  req@ffff8cd0c3da9e00 x1874671626692864/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826998 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826973.782546:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626692864/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826998 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826973.782551:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826973.782553:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826973.782555:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626692864:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826973.782559:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626692992:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787826973.782571:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826973.782576:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826973/real 1787826973]  req@ffff8cd0c3da92c0 x1874671626692992/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826998 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826973.782583:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626692992/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787826998 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826973.782587:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826973.782588:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826973.782590:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626692992:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826973.782593:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626693120:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826973.782603:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826973.782608:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826973/real 1787826973]  req@ffff8cd0c3daa580 x1874671626693120/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826998 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826973.782653:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626693120/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826998 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826973.782661:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826973.782664:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826973.782668:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626693120:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787826973.974184:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826974.358276:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787826974.998201:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826975.382253:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787826976.022184:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826976.406222:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787826977.046211:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826977.430349:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787826978.070184:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826978.454238:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787826978.902229:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826978.902239:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787826978.902243:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826978.902252:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 864
00000100:00080000:3.0:1787826978.902262:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826978.902269:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787826978.902305:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626693248/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826978.902335:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826978.902338:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826978.902340:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826978.902344:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 864
00000100:00080000:3.0:1787826978.902347:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826978.902352:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826978.902361:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826978.902374:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626693376/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826978.902384:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626693248:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787826978.902388:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826978.902393:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787826978.902395:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826978.902399:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 864
00000100:00080000:3.0:1787826978.902403:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826978.902423:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826978.902429:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826978.902441:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626693504/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787826978.902447:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787826978.902456:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826978.902460:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826978.902462:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826978.902467:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 864
00000100:00080000:3.0:1787826978.902472:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826978.902476:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826978.902481:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826978.902492:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826978/real 1787826978]  req@ffff8cd0c8ead2c0 x1874671626693248/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827003 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826978.902494:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626693632/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826978.902530:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626693248/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827003 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826978.902541:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826978.902545:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826978.902548:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626693248:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826978.902569:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626693376:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826978.902605:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826978.902618:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826978/real 1787826978]  req@ffff8cd0c8eac780 x1874671626693376/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827003 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826978.902630:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626693376/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827003 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826978.902649:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826978.902653:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826978.902657:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626693376:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826978.902663:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626693504:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787826978.902685:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826978.902698:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826978/real 1787826978]  req@ffff8cd0c8eae1c0 x1874671626693504/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827003 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826978.902720:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626693504/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827003 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826978.902729:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826978.902732:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826978.902735:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626693504:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826978.902740:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626693632:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826978.902766:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826978.902778:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826978/real 1787826978]  req@ffff8cd0c8eac3c0 x1874671626693632/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827003 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826978.902789:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626693632/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827003 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826978.902796:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826978.902799:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826978.902808:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626693632:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787826979.094194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826979.478268:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787826980.118143:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826980.502254:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787826981.142266:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826981.526229:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787826982.166177:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826982.550284:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787826983.190202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826983.574339:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787826984.022685:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826984.022695:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787826984.022715:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826984.022724:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 869
00000100:00080000:2.0:1787826984.022735:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826984.022741:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787826984.022775:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daad00 x1874671626693760/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826984.022806:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1787826984.022915:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626693760:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787826984.023046:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826984.023049:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826984.023052:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 869
00000100:00080000:2.0:1787826984.023056:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826984.023059:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826984.023067:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826984.023078:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626693888/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826984.023108:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826984.023111:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787826984.023112:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826984.023115:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 869
00000100:00080000:2.0:1787826984.023117:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826984.023119:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826984.023123:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826984.023131:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626694016/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787826984.023146:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787826984.023148:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787826984.023149:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787826984.023151:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 869
00000100:00080000:2.0:1787826984.023154:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787826984.023158:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787826984.023162:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787826984.023173:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626694144/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826984.023278:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626693888:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826984.023308:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626694016:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826984.023342:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626694144:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787826984.214189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826984.598266:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787826985.239817:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826985.622296:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787826986.262226:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826986.646580:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:1.0:1787826987.094601:0:2346:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:1.0:1787826987.094643:0:2346:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1787826987.094672:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:0.0:1787826987.095966:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826984/real 1787826987]  req@ffff8cd0c3daad00 x1874671626693760/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827009 ref 2 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826987.096009:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daad00 x1874671626693760/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827009 ref 2 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826987.096022:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826987.096027:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826987.096032:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626693760:192.168.203.145@tcp:250:kworker.0
00000800:00000100:1.0:1787826987.096625:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787826987.096661:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787826987.096670:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:0.0:1787826987.096957:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826984/real 1787826987]  req@ffff8cd0c3daa580 x1874671626693888/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827009 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826987.096975:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626693888/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827009 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826987.096985:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826987.096988:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826987.096991:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626693888:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826987.096998:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826984/real 1787826987]  req@ffff8cd0c3da92c0 x1874671626694016/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827009 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826987.097028:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626694016/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827009 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826987.097036:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826987.097039:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826987.097043:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626694016:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826987.097049:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826984/real 1787826987]  req@ffff8cd0c3da9e00 x1874671626694144/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827009 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826987.097059:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626694144/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827009 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826987.097066:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826987.097069:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826987.097072:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626694144:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787826987.286210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826987.670558:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787826988.310223:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826988.694490:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787826989.142216:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826989.142234:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787826989.142238:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826989.142248:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 874
00000100:00080000:3.0:1787826989.142260:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826989.142267:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787826989.142299:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626694272/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826989.142332:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826989.142335:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826989.142349:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826989.142354:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 874
00000100:00080000:3.0:1787826989.142358:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826989.142363:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826989.142373:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826989.142386:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626694400/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826989.142392:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626694272:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787826989.142401:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826989.142406:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787826989.142408:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826989.142412:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 874
00000100:00080000:3.0:1787826989.142415:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826989.142419:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826989.142425:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826989.142438:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626694528/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826989.142447:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826989.142450:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826989.142452:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1787826989.142458:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787826989.142458:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 874
00000100:00080000:3.0:1787826989.142465:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826989.142470:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826989.142475:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826989.142506:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826989/real 1787826989]  req@ffff8cd0c8eac000 x1874671626694272/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827014 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826989.142520:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626694656/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787826989.142550:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626694272/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827014 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826989.142562:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826989.142566:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826989.142576:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626694272:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826989.142601:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626694400:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826989.142623:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826989.142634:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826989/real 1787826989]  req@ffff8cd0c8eacf00 x1874671626694400/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827014 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826989.142646:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626694400/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827014 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826989.142654:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826989.142657:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826989.142660:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626694400:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826989.142666:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626694528:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787826989.142680:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826989.142689:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826989/real 1787826989]  req@ffff8cd0c8eac3c0 x1874671626694528/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827014 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826989.142699:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626694528/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827014 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826989.142707:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826989.142710:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826989.142713:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626694528:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826989.142718:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626694656:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826989.142732:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826989.142748:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826989/real 1787826989]  req@ffff8cd0c8eae1c0 x1874671626694656/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827014 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826989.142759:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626694656/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827014 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826989.142767:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826989.142770:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826989.142773:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626694656:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787826989.334226:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826989.718216:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787826990.358185:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826990.742280:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787826991.382210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826991.766276:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787826992.406196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826992.790325:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787826993.430207:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826993.814267:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787826994.262242:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826994.262252:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787826994.262264:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826994.262274:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 880
00000100:00080000:0.0:1787826994.262284:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826994.262290:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787826994.262329:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626694784/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826994.262388:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626694784:192.168.203.145@tcp:250:kworker.0
00000800:00000100:3.0:1787826994.262455:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787826994.262492:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826994.262496:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826994.262499:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826994.262503:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 880
00000100:00080000:0.0:1787826994.262507:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826994.262511:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826994.262521:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826994.262547:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626694912/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826994.262810:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826994/real 1787826994]  req@ffff8cd0c8e9fc00 x1874671626694784/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827019 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826994.262827:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626694784/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827019 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826994.262837:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826994.262841:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826994.262844:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626694784:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826994.262865:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626694912:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826994.262887:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787826994.262976:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826994.262980:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787826994.262982:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826994.262986:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 880
00000100:00100000:3.0:1787826994.263008:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826994/real 1787826994]  req@ffff8cd0c8e9de00 x1874671626694912/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827019 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826994.263021:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626694912/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827019 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826994.263029:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826994.263032:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826994.263035:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626694912:192.168.203.145@tcp:38:kworker.0
00000100:00080000:0.0:1787826994.263078:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826994.263083:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826994.263110:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826994.263153:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626695040/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787826994.263172:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787826994.263178:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787826994.263180:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787826994.263184:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 880
00000100:00080000:0.0:1787826994.263188:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787826994.263192:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787826994.263197:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826994.263209:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626695168/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787826994.263216:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626695040:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787826994.263259:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826994.263273:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826994/real 1787826994]  req@ffff8cd0c8e9e580 x1874671626695040/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827019 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826994.263285:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626695040/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827019 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826994.263292:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826994.263295:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826994.263297:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626695040:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787826994.263309:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626695168:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787826994.263327:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787826994.263337:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826994/real 1787826994]  req@ffff8cd0c8e9e940 x1874671626695168/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827019 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787826994.263348:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626695168/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827019 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787826994.263356:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787826994.263359:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787826994.263362:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626695168:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787826994.456026:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826994.838527:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787826995.478205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826995.862254:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787826996.502203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826996.886592:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787826997.526188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826997.910253:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787826998.550205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787826998.934298:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787826999.382297:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826999.382303:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787826999.382307:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826999.382313:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 885
00000100:00080000:3.0:1787826999.382322:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826999.382339:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787826999.382365:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626695296/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826999.382386:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826999.382388:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826999.382389:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826999.382391:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 885
00000100:00080000:3.0:1787826999.382394:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826999.382396:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826999.382403:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826999.382411:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626695424/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826999.382418:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826999.382419:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787826999.382421:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826999.382423:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 885
00000100:00080000:3.0:1787826999.382424:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826999.382426:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826999.382429:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787826999.382434:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626695296:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787826999.382436:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626695552/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787826999.382458:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787826999.382464:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787826999.382471:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787826999.382473:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 885
00000100:00080000:3.0:1787826999.382477:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787826999.382486:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787826999.382494:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787826999.382506:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626695680/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787826999.382524:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826999.382562:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826999/real 1787826999]  req@ffff8cd0c8eac780 x1874671626695296/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827024 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826999.382583:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626695296/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827024 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826999.382593:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826999.382596:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826999.382600:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626695296:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787826999.382606:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626695424:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826999.382630:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826999.382641:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826999/real 1787826999]  req@ffff8cd0c8ead2c0 x1874671626695424/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827024 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826999.382653:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626695424/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827024 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826999.382659:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826999.382662:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826999.382664:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626695424:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787826999.382687:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626695552:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787826999.382717:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826999.382729:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826999/real 1787826999]  req@ffff8cd0c8eaf840 x1874671626695552/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827024 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826999.382753:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626695552/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827024 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826999.382759:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826999.382762:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826999.382764:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626695552:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787826999.382770:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626695680:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787826999.382791:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787826999.382802:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826999/real 1787826999]  req@ffff8cd0c8eada40 x1874671626695680/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827024 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787826999.382812:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626695680/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827024 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787826999.382819:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787826999.382822:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787826999.382825:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626695680:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787826999.574249:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787826999.958230:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827000.598209:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827000.982380:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827001.623172:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827002.006245:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827002.646214:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827003.030403:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827003.670209:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827004.054295:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787827004.505242:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827004.505260:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787827004.505265:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827004.505274:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 890
00000100:00080000:0.0:1787827004.505285:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827004.505304:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787827004.505332:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626695808/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827004.505355:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827004.505358:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827004.505360:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827004.505363:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 890
00000100:00080000:0.0:1787827004.505367:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827004.505370:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827004.505380:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827004.505390:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626695936/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827004.505396:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626695808:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787827004.505404:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827004.505418:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787827004.505420:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827004.505424:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 890
00000100:00080000:0.0:1787827004.505427:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827004.505430:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827004.505435:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827004.505446:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626696064/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827004.505456:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827004.505459:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827004.505466:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:3.0:1787827004.505467:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787827004.505473:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 890
00000100:00080000:0.0:1787827004.505479:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827004.505484:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827004.505489:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827004.505501:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626696192/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827004.505512:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827004/real 1787827004]  req@ffff8cd0c8e9cf00 x1874671626695808/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827029 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827004.505531:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626695808/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827029 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827004.505541:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827004.505546:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827004.505550:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626695808:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827004.505573:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626695936:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827004.505595:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827004.505608:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827004/real 1787827004]  req@ffff8cd0c8e9d680 x1874671626695936/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827029 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827004.505620:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626695936/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827029 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827004.505639:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827004.505643:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827004.505646:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626695936:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827004.505652:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626696064:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827004.505675:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827004.505687:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827004/real 1787827004]  req@ffff8cd0c8e9ed00 x1874671626696064/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827029 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827004.505698:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626696064/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827029 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827004.505705:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827004.505708:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827004.505711:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626696064:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827004.505725:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626696192:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827004.505745:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827004.505756:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827004/real 1787827004]  req@ffff8cd0c8e9f840 x1874671626696192/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827029 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827004.505768:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626696192/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827029 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827004.505775:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827004.505778:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827004.505787:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626696192:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827004.694244:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827005.078710:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827005.719018:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827006.102265:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827006.742204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827007.126960:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827007.766210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827008.150222:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827008.790398:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827009.174275:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787827009.622400:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827009.622410:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787827009.622414:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827009.622424:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 895
00000100:00080000:0.0:1787827009.622434:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827009.622441:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787827009.622480:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626696320/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827009.622511:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827009.622515:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827009.622517:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827009.622531:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 895
00000100:00080000:0.0:1787827009.622534:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827009.622538:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827009.622548:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827009.622587:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626696448/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827009.622597:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827009.622600:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787827009.622601:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827009.622604:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 895
00000100:00080000:0.0:1787827009.622608:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827009.622612:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827009.622617:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827009.622627:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626696576/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827009.622636:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827009.622639:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827009.622641:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827009.622645:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 895
00000100:00080000:0.0:1787827009.622648:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1787827009.622652:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626696320:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787827009.622654:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827009.622663:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827009.622685:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626696704/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787827009.623355:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827009.623406:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827009/real 1787827009]  req@ffff8cd0c8e9c000 x1874671626696320/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827034 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827009.623423:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626696320/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827034 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827009.623433:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827009.623437:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827009.623440:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626696320:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827009.623448:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626696448:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827009.623473:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827009.623487:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827009/real 1787827009]  req@ffff8cd0c8e9f480 x1874671626696448/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827034 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827009.623498:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626696448/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827034 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827009.623505:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827009.623508:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827009.623511:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626696448:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827009.623517:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626696576:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827009.623538:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827009.623562:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827009/real 1787827009]  req@ffff8cd0c8e9cb40 x1874671626696576/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827034 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827009.623574:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626696576/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827034 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827009.623582:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827009.623585:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827009.623588:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626696576:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827009.623615:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626696704:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827009.623636:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827009.623647:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827009/real 1787827009]  req@ffff8cd0c8e9c3c0 x1874671626696704/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827034 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827009.623669:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626696704/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827034 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827009.623677:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827009.623680:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827009.623683:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626696704:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827009.814337:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827010.198226:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827010.838216:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827011.222273:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827011.862272:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827012.246279:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827012.886200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827013.270223:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827013.910207:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827014.294220:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787827014.742257:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827014.742293:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787827014.742299:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827014.742311:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 900
00000100:00080000:0.0:1787827014.742323:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827014.742330:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787827014.742380:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d2c0 x1874671626696832/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827014.742404:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827014.742406:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827014.742408:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827014.742410:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 900
00000100:00080000:0.0:1787827014.742413:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827014.742415:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827014.742433:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827014.742448:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626696960/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827014.742460:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787827014.742463:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626696832:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787827014.742465:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787827014.742469:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827014.742475:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 900
00000100:00080000:0.0:1787827014.742478:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827014.742480:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827014.742485:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827014.742495:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626697088/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827014.742507:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827014.742510:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827014.742511:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827014.742515:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 900
00000100:00080000:0.0:1787827014.742524:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827014.742527:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827014.742531:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827014.742543:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626697216/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787827014.742549:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827014.742604:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827014/real 1787827014]  req@ffff8cd0c8e9d2c0 x1874671626696832/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827039 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827014.742622:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d2c0 x1874671626696832/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827039 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827014.742640:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827014.742644:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827014.742647:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626696832:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827014.742675:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626696960:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827014.742711:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827014.742725:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827014/real 1787827014]  req@ffff8cd0c8e9c3c0 x1874671626696960/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827039 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827014.742737:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626696960/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827039 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827014.742745:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827014.742749:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827014.742751:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626696960:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827014.742756:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626697088:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827014.742772:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827014.742779:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827014/real 1787827014]  req@ffff8cd0c8e9cb40 x1874671626697088/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827039 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827014.742785:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626697088/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827039 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827014.742791:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827014.742793:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827014.742795:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626697088:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827014.742804:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626697216:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827014.742817:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827014.742823:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827014/real 1787827014]  req@ffff8cd0c8e9f480 x1874671626697216/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827039 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827014.742830:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626697216/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827039 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827014.742834:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827014.742836:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827014.742837:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626697216:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827014.934168:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827015.318291:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827015.958198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827016.342268:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827016.985474:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827017.366548:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827018.006283:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827018.390213:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827019.030203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827019.414261:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827019.862216:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827019.862225:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827019.862229:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827019.862239:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 905
00000100:00080000:2.0:1787827019.862250:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827019.862258:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827019.862294:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626697344/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827019.862331:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827019.862334:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827019.862338:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827019.862341:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 905
00000100:00080000:2.0:1787827019.862344:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827019.862347:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827019.862370:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827019.862385:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626697472/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827019.862385:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626697344:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787827019.862402:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827019.862404:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827019.862406:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827019.862409:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 905
00000100:00080000:2.0:1787827019.862414:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827019.862417:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827019.862422:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827019.862434:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626697600/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827019.862443:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827019.862446:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827019.862448:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827019.862452:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 905
00000100:00080000:2.0:1787827019.862455:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1787827019.862460:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787827019.862462:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827019.862468:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827019.862481:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626697728/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827019.862505:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827019/real 1787827019]  req@ffff8cd0c3da8000 x1874671626697344/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827044 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827019.862525:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626697344/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827044 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827019.862541:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827019.862545:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827019.862549:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626697344:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827019.862569:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626697472:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827019.862590:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827019.862603:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827019/real 1787827019]  req@ffff8cd0c3dab840 x1874671626697472/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827044 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827019.862622:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626697472/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827044 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827019.862630:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827019.862633:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827019.862641:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626697472:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827019.862647:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626697600:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827019.862668:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827019.862678:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827019/real 1787827019]  req@ffff8cd0c3da8780 x1874671626697600/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827044 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827019.862690:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626697600/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827044 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827019.862697:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827019.862700:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827019.862703:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626697600:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827019.862709:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626697728:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827019.862724:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827019.862734:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827019/real 1787827019]  req@ffff8cd0c3dabc00 x1874671626697728/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827044 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827019.862745:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626697728/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827044 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827019.862776:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827019.862779:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827019.862783:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626697728:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827020.054200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827020.438226:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827021.078213:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827021.462306:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827022.102194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827022.486744:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827023.126191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827023.510250:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827024.150188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827024.534234:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827024.982236:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827024.982249:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827024.982284:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827024.982294:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 910
00000100:00080000:2.0:1787827024.982310:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827024.982316:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827024.982358:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626697856/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827024.982391:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827024.982395:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827024.982397:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827024.982401:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 910
00000100:00080000:2.0:1787827024.982405:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827024.982409:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827024.982421:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827024.982434:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626697984/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827024.982444:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827024.982448:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827024.982450:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827024.982454:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 910
00000100:00080000:2.0:1787827024.982458:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1787827024.982463:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626697856:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787827024.982466:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827024.982476:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827024.982489:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626698112/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827024.982514:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827024.982518:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827024.982520:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827024.982524:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 910
00000100:00080000:2.0:1787827024.982529:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827024.982534:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827024.982540:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787827024.982548:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827024.982555:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626698240/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827024.982601:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827024/real 1787827024]  req@ffff8cd0c3daa1c0 x1874671626697856/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827049 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827024.982631:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626697856/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827049 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827024.982643:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827024.982647:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827024.982651:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626697856:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827024.982659:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626697984:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827024.982681:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827024.982693:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827024/real 1787827024]  req@ffff8cd0c3da9a40 x1874671626697984/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827049 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827024.982704:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626697984/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827049 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827024.982712:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827024.982716:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827024.982719:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626697984:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827024.982754:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626698112:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827024.982778:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827024.982790:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827024/real 1787827024]  req@ffff8cd0c3dab0c0 x1874671626698112/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827049 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827024.982802:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626698112/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827049 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827024.982822:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827024.982826:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827024.982830:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626698112:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827024.982835:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626698240:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827024.982855:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827024.982865:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827024/real 1787827024]  req@ffff8cd0c3daa940 x1874671626698240/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827049 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827024.982876:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626698240/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827049 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827024.982884:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827024.982887:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827024.982890:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626698240:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827025.174206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827025.558297:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827026.198214:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827026.582305:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827027.222191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827027.606269:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827028.246205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827028.630266:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827029.270151:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827029.654281:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827030.102193:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827030.102199:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827030.102203:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827030.102210:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 915
00000100:00080000:2.0:1787827030.102218:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827030.102223:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827030.102267:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626698368/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827030.102288:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827030.102290:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827030.102291:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827030.102293:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 915
00000100:00080000:2.0:1787827030.102296:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827030.102298:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827030.102317:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827030.102326:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626698496/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827030.102334:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787827030.102337:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626698368:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787827030.102338:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827030.102341:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827030.102347:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 915
00000100:00080000:2.0:1787827030.102349:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827030.102352:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827030.102355:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827030.102363:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626698624/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827030.102368:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827030.102374:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827030.102375:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827030.102378:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 915
00000100:00080000:2.0:1787827030.102387:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827030.102389:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827030.102392:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787827030.102399:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827030.102401:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab480 x1874671626698752/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827030.102433:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827030/real 1787827030]  req@ffff8cd0c3da83c0 x1874671626698368/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827055 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827030.102452:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626698368/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827055 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827030.102460:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827030.102462:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827030.102465:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626698368:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827030.102480:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626698496:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827030.102495:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827030.102502:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827030/real 1787827030]  req@ffff8cd0c3da9680 x1874671626698496/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827055 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827030.102509:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626698496/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827055 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827030.102514:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827030.102516:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827030.102518:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626698496:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827030.102521:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626698624:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827030.102530:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827030.102535:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827030/real 1787827030]  req@ffff8cd0c3da8f00 x1874671626698624/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827055 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827030.102541:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626698624/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827055 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827030.102553:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827030.102555:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827030.102557:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626698624:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827030.102561:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626698752:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827030.102572:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827030.102577:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827030/real 1787827030]  req@ffff8cd0c3dab480 x1874671626698752/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827055 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827030.102583:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab480 x1874671626698752/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827055 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827030.102588:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827030.102589:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827030.102590:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626698752:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827030.294236:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827030.678294:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827031.318262:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827031.702259:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827032.342270:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827032.726270:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827033.366225:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827033.750321:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827034.390679:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827034.774190:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827035.222186:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827035.222223:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827035.222240:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827035.222247:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 921
00000100:00080000:2.0:1787827035.222255:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827035.222260:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827035.222287:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8b40 x1874671626698880/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827035.222309:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827035.222312:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827035.222313:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827035.222315:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 921
00000100:00080000:2.0:1787827035.222317:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827035.222319:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827035.222326:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827035.222335:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab480 x1874671626699008/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827035.222341:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827035.222344:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827035.222345:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827035.222347:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 921
00000100:00080000:2.0:1787827035.222349:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827035.222353:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827035.222355:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626698880:192.168.203.145@tcp:250:kworker.0
00000008:00100000:2.0:1787827035.222357:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827035.222366:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626699136/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827035.222381:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827035.222384:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827035.222385:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827035.222387:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 921
00000100:00080000:2.0:1787827035.222400:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827035.222403:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827035.222407:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787827035.222410:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827035.222419:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626699264/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827035.222445:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827035/real 1787827035]  req@ffff8cd0c3da8b40 x1874671626698880/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827060 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827035.222464:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8b40 x1874671626698880/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827060 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827035.222471:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827035.222474:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827035.222476:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626698880:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827035.222487:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626699008:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827035.222503:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827035.222511:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827035/real 1787827035]  req@ffff8cd0c3dab480 x1874671626699008/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827060 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827035.222519:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab480 x1874671626699008/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827060 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827035.222524:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827035.222526:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827035.222528:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626699008:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827035.222544:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626699136:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827035.222554:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827035.222560:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827035/real 1787827035]  req@ffff8cd0c3da8f00 x1874671626699136/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827060 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827035.222566:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626699136/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827060 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827035.222571:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827035.222572:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827035.222574:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626699136:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827035.222577:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626699264:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827035.222586:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827035.222591:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827035/real 1787827035]  req@ffff8cd0c3da9680 x1874671626699264/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827060 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827035.222602:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626699264/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827060 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827035.222607:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827035.222609:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827035.222611:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626699264:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827035.414180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827035.798256:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827036.438164:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827036.822246:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827037.462173:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827037.846272:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827038.486144:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827038.870234:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827039.510206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827039.894248:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827040.342187:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827040.342210:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827040.342214:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827040.342222:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 926
00000100:00080000:1.0:1787827040.342231:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827040.342238:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827040.342276:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626699392/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827040.342305:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827040.342309:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827040.342311:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827040.342314:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 926
00000100:00080000:1.0:1787827040.342318:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827040.342322:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827040.342331:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827040.342344:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626699520/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827040.342351:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626699392:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787827040.342357:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827040.342361:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827040.342363:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827040.342368:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 926
00000100:00080000:1.0:1787827040.342372:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827040.342375:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827040.342381:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827040.342391:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626699648/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827040.342402:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827040.342404:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827040.342406:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827040.342411:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 926
00000800:00000100:3.0:1787827040.342411:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787827040.342417:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827040.342427:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827040.342433:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827040.342446:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626699776/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827040.342447:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827040/real 1787827040]  req@ffff8cd0d48f70c0 x1874671626699392/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827065 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827040.342479:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626699392/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827065 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827040.342488:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827040.342491:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827040.342494:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626699392:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827040.342510:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626699520:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827040.342525:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827040.342532:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827040/real 1787827040]  req@ffff8cd0d48f43c0 x1874671626699520/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827065 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827040.342540:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626699520/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827065 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827040.342554:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827040.342558:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827040.342561:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626699520:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827040.342567:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626699648:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827040.342587:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827040.342597:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827040/real 1787827040]  req@ffff8cd0d48f5a40 x1874671626699648/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827065 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827040.342608:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626699648/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827065 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827040.342613:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827040.342615:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827040.342617:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626699648:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827040.342621:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626699776:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827040.342633:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827040.342639:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827040/real 1787827040]  req@ffff8cd0d48f7840 x1874671626699776/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827065 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827040.342646:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626699776/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827065 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827040.342650:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827040.342652:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827040.342658:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626699776:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827040.534210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827040.918325:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827041.558211:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827041.942275:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827042.582221:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827042.966278:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827043.606196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827043.990293:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827044.630239:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827045.014288:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827045.462261:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827045.462270:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827045.462273:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827045.462281:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 931
00000100:00080000:2.0:1787827045.462289:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827045.462294:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827045.462324:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626699904/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827045.462354:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827045.462356:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827045.462358:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827045.462360:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 931
00000100:00080000:2.0:1787827045.462362:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827045.462364:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827045.462382:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827045.462392:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626700032/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827045.462403:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626699904:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787827045.462406:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827045.462411:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827045.462412:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827045.462415:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 931
00000100:00080000:2.0:1787827045.462418:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827045.462420:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827045.462424:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827045.462431:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626700160/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827045.462437:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827045.462440:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827045.462442:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827045.462445:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 931
00000100:00080000:2.0:1787827045.462449:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827045.462453:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827045.462458:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827045.462470:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626700288/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787827045.462480:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827045.462522:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827045/real 1787827045]  req@ffff8cd0c3da83c0 x1874671626699904/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827070 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827045.462540:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626699904/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827070 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827045.462549:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827045.462552:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827045.462555:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626699904:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827045.462578:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626700032:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827045.462599:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827045.462612:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827045/real 1787827045]  req@ffff8cd0c3daa940 x1874671626700032/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827070 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827045.462624:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626700032/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827070 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827045.462632:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827045.462636:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827045.462639:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626700032:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827045.462645:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626700160:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827045.462661:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827045.462676:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827045/real 1787827045]  req@ffff8cd0c3dab0c0 x1874671626700160/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827070 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827045.462683:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626700160/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827070 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827045.462688:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827045.462689:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827045.462691:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626700160:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827045.462695:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626700288:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827045.462705:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827045.462711:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827045/real 1787827045]  req@ffff8cd0c3da9a40 x1874671626700288/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827070 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827045.462717:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626700288/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827070 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827045.462721:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827045.462723:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827045.462725:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626700288:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827045.654257:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827046.038307:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827046.678197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827047.062238:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827047.702195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827048.086271:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827048.726218:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827049.110274:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827049.750198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827050.135068:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827050.582172:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827050.582180:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827050.582203:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827050.582211:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 936
00000100:00080000:2.0:1787827050.582219:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827050.582225:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827050.582251:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626700416/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827050.582271:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827050.582273:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827050.582274:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827050.582276:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 936
00000100:00080000:2.0:1787827050.582278:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827050.582281:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827050.582288:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827050.582295:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626700544/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827050.582302:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827050.582304:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827050.582305:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827050.582307:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 936
00000100:00080000:2.0:1787827050.582309:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827050.582311:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827050.582315:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827050.582316:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626700416:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827050.582322:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626700672/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827050.582339:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827050.582341:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827050.582342:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827050.582345:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 936
00000100:00080000:2.0:1787827050.582348:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827050.582350:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827050.582354:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827050.582361:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626700800/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827050.582407:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626700544:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827050.582436:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626700672:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827050.582460:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626700800:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827050.774219:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827051.159222:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827051.798228:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827052.182243:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827052.822241:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827053.206228:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:1.0:1787827053.654355:0:2343:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:1.0:1787827053.654393:0:2343:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1787827053.654408:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827053.654441:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827053.654448:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827053.654455:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:3.0:1787827053.654493:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827050/real 1787827053]  req@ffff8cd0c3da8780 x1874671626700672/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827075 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827053.654518:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626700672/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827075 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827053.654530:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827053.654534:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827053.654538:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626700672:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827053.654545:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827050/real 1787827053]  req@ffff8cd0c3dab840 x1874671626700800/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827075 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827053.654563:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626700800/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827075 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827053.654571:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827053.654573:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827053.654575:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626700800:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827053.654579:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827050/real 1787827053]  req@ffff8cd0c3daa1c0 x1874671626700416/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827075 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827053.654589:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626700416/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827075 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827053.654596:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827053.654599:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827053.654603:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626700416:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827053.654623:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827050/real 1787827053]  req@ffff8cd0c3dabc00 x1874671626700544/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827075 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827053.654635:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626700544/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827075 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827053.654642:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827053.654645:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827053.654648:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626700544:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827053.846986:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827054.230321:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827054.870199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827055.254296:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827055.702255:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827055.702268:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827055.702273:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827055.702285:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 941
00000100:00080000:2.0:1787827055.702298:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827055.702306:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827055.702348:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626700928/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827055.702387:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827055.702398:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827055.702400:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827055.702404:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 941
00000100:00080000:2.0:1787827055.702408:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827055.702412:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827055.702448:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827055.702461:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626701056/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827055.702472:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827055.702475:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827055.702477:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827055.702481:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 941
00000100:00080000:2.0:1787827055.702485:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827055.702489:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827055.702494:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827055.702503:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626701184/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827055.702512:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827055.702515:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827055.702517:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827055.702521:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 941
00000100:00080000:2.0:1787827055.702524:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827055.702528:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827055.702532:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827055.702542:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626701312/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827055.702679:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626700928:192.168.203.145@tcp:250:kworker.0
00000800:00000100:3.0:1787827055.702755:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827055.702805:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827055/real 1787827055]  req@ffff8cd0c3da8000 x1874671626700928/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827080 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827055.702824:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626700928/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827080 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827055.702841:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827055.702845:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827055.702849:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626700928:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827055.702856:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626701056:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827055.702878:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827055.702889:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827055/real 1787827055]  req@ffff8cd0c3da9e00 x1874671626701056/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827080 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827055.702899:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626701056/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827080 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827055.702914:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827055.702918:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827055.702921:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626701056:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827055.702927:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626701184:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827055.702949:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827055.702960:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827055/real 1787827055]  req@ffff8cd0c3da92c0 x1874671626701184/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827080 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827055.702970:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626701184/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827080 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827055.702978:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827055.702981:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827055.702984:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626701184:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827055.702989:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626701312:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827055.703141:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827055.703153:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827055/real 1787827055]  req@ffff8cd0c3daa580 x1874671626701312/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827080 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827055.703167:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626701312/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827080 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827055.703174:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827055.703178:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827055.703188:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626701312:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827055.894252:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827056.278252:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827056.918170:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827057.302345:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827057.942301:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827058.326296:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827058.966356:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827059.350713:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827059.990238:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827060.375376:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827060.822213:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827060.822222:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827060.822226:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827060.822236:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 946
00000100:00080000:1.0:1787827060.822247:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827060.822254:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827060.822289:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626701440/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827060.822324:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827060.822328:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827060.822330:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827060.822333:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 946
00000100:00080000:1.0:1787827060.822351:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827060.822356:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827060.822366:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827060.822376:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626701440:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787827060.822382:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626701568/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827060.822395:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827060.822398:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827060.822400:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827060.822404:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 946
00000100:00080000:1.0:1787827060.822409:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827060.822416:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827060.822422:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827060.822433:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626701696/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787827060.822438:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787827060.822448:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827060.822452:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827060.822454:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827060.822458:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 946
00000100:00080000:1.0:1787827060.822462:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827060.822466:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827060.822471:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827060.822480:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827060/real 1787827060]  req@ffff8cd0d48f4000 x1874671626701440/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827085 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787827060.822492:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626701824/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827060.822500:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626701440/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827085 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827060.822516:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827060.822520:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827060.822524:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626701440:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827060.822549:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626701568:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827060.822573:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827060.822585:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827060/real 1787827060]  req@ffff8cd0d48f6580 x1874671626701568/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827085 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827060.822596:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626701568/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827085 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827060.822603:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827060.822606:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827060.822609:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626701568:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827060.822614:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626701696:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827060.822630:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827060.822747:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827060/real 1787827060]  req@ffff8cd0d48f7480 x1874671626701696/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827085 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827060.822759:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626701696/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827085 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827060.822766:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827060.822769:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827060.822772:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626701696:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827060.822777:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626701824:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827060.822797:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827060.822806:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827060/real 1787827060]  req@ffff8cd0d48f4b40 x1874671626701824/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827085 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827060.822817:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626701824/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827085 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827060.822824:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827060.822827:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827060.822830:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626701824:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827061.014193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827061.398291:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827062.038210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827062.422267:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827063.062261:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827063.448404:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827064.086180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827064.470460:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827065.110196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827065.494310:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827065.942506:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827065.942515:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827065.942519:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827065.942529:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 951
00000100:00080000:1.0:1787827065.942540:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827065.942546:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827065.942583:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626701952/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827065.942614:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827065.942617:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827065.942619:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827065.942622:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 951
00000100:00080000:1.0:1787827065.942626:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827065.942629:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827065.942637:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827065.942647:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626702080/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827065.942655:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827065.942658:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827065.942671:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827065.942675:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 951
00000100:00080000:1.0:1787827065.942678:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827065.942681:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827065.942686:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827065.942693:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626702208/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827065.942701:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827065.942704:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827065.942706:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827065.942709:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 951
00000100:00080000:1.0:1787827065.942712:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827065.942715:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827065.942719:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827065.942726:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626702336/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827065.942875:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626701952:192.168.203.145@tcp:250:kworker.0
00000800:00000100:3.0:1787827065.942931:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827065.942969:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827065/real 1787827065]  req@ffff8cd0d48f4b40 x1874671626701952/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827090 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827065.942999:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626701952/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827090 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827065.943016:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827065.943020:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827065.943023:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626701952:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827065.943030:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626702080:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827065.943055:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827065.943069:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827065/real 1787827065]  req@ffff8cd0d48f7480 x1874671626702080/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827090 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827065.943081:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626702080/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827090 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827065.943110:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827065.943114:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827065.943118:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626702080:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827065.943124:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626702208:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827065.943146:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827065.943157:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827065/real 1787827065]  req@ffff8cd0d48f6580 x1874671626702208/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827090 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827065.943168:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626702208/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827090 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827065.943176:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827065.943179:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827065.943182:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626702208:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827065.943206:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626702336:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827065.943228:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827065.943239:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827065/real 1787827065]  req@ffff8cd0d48f4000 x1874671626702336/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827090 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827065.943251:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626702336/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827090 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827065.943259:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827065.943262:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827065.943266:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626702336:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827066.134207:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827066.518273:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827067.158261:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827067.542240:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827068.182209:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827068.566271:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827069.206204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827069.590227:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827070.230230:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827070.614256:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827071.062177:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827071.062184:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827071.062187:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827071.062195:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 956
00000100:00080000:1.0:1787827071.062218:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827071.062223:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827071.062249:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626702464/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827071.062270:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827071.062274:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827071.062276:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827071.062280:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 956
00000100:00080000:1.0:1787827071.062283:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827071.062287:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827071.062296:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827071.062308:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626702592/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827071.062314:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626702464:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787827071.062321:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827071.062325:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827071.062327:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827071.062332:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 956
00000100:00080000:1.0:1787827071.062336:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827071.062339:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827071.062344:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827071.062355:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626702720/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827071.062364:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827071.062373:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827071.062376:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827071.062380:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 956
00000100:00080000:1.0:1787827071.062385:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1787827071.062387:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787827071.062392:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827071.062398:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827071.062422:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626702848/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827071.062427:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827071/real 1787827071]  req@ffff8cd0d48f7840 x1874671626702464/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827096 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827071.062452:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626702464/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827096 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827071.062461:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827071.062465:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827071.062469:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626702464:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827071.062490:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626702592:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827071.062511:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827071.062523:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827071/real 1787827071]  req@ffff8cd0d48f5a40 x1874671626702592/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827096 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827071.062542:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626702592/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827096 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827071.062551:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827071.062554:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827071.062557:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626702592:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827071.062563:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626702720:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827071.062583:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827071.062594:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827071/real 1787827071]  req@ffff8cd0d48f43c0 x1874671626702720/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827096 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827071.062605:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626702720/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827096 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827071.062613:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827071.062616:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827071.062628:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626702720:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827071.062634:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626702848:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827071.062648:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827071.062655:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827071/real 1787827071]  req@ffff8cd0d48f70c0 x1874671626702848/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827096 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827071.062664:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626702848/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827096 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827071.062682:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827071.062694:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827071.062698:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626702848:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827071.254165:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827071.638210:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827072.278204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827072.662243:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827073.302191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827073.686222:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827074.326207:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827074.710266:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827075.350206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827075.734228:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827076.182203:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827076.182213:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827076.182217:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827076.182226:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 962
00000100:00080000:1.0:1787827076.182237:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827076.182244:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827076.182278:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626702976/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827076.182306:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827076.182310:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827076.182311:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827076.182315:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 962
00000100:00080000:1.0:1787827076.182331:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827076.182336:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827076.182345:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827076.182349:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626702976:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787827076.182361:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626703104/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827076.182372:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827076.182376:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827076.182378:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827076.182381:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 962
00000100:00080000:1.0:1787827076.182385:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827076.182388:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827076.182393:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827076.182404:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626703232/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787827076.182410:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787827076.182420:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827076.182424:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827076.182426:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827076.182430:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 962
00000100:00080000:1.0:1787827076.182434:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827076.182438:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827076.182443:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827076.182450:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827076/real 1787827076]  req@ffff8cd0d48f61c0 x1874671626702976/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827101 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787827076.182456:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626703360/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827076.182477:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626702976/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827101 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827076.182496:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827076.182499:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827076.182502:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626702976:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827076.182520:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626703104:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827076.182538:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827076.182546:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827076/real 1787827076]  req@ffff8cd0d48f7c00 x1874671626703104/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827101 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827076.182553:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626703104/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827101 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827076.182558:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827076.182560:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827076.182562:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626703104:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827076.182571:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626703232:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827076.182585:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827076.182591:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827076/real 1787827076]  req@ffff8cd0d48f52c0 x1874671626703232/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827101 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827076.182598:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626703232/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827101 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827076.182602:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827076.182604:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827076.182605:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626703232:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827076.182608:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626703360:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827076.182617:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827076.182622:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827076/real 1787827076]  req@ffff8cd0d48f6d00 x1874671626703360/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827101 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827076.182635:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626703360/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827101 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827076.182642:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827076.182645:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827076.182648:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626703360:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827076.374199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827076.758232:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827077.398198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827077.782252:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827078.422193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827078.806264:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827079.446182:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827079.830261:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827080.470129:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827080.854232:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827081.302245:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827081.302258:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827081.302263:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827081.302276:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 967
00000100:00080000:1.0:1787827081.302290:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827081.302299:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827081.302364:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626703488/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827081.302397:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827081.302400:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827081.302403:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827081.302407:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 967
00000100:00080000:1.0:1787827081.302410:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827081.302414:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827081.302426:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827081.302441:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626703616/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827081.302451:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827081.302454:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827081.302474:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827081.302479:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 967
00000100:00080000:1.0:1787827081.302484:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827081.302488:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827081.302495:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827081.302498:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626703488:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787827081.302521:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626703744/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827081.302534:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827081.302537:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827081.302539:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827081.302544:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 967
00000100:00080000:1.0:1787827081.302548:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827081.302552:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827081.302557:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827081.302569:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626703872/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787827081.302583:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827081.302650:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827081/real 1787827081]  req@ffff8cd0d48f6d00 x1874671626703488/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827106 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827081.302699:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626703488/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827106 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827081.302711:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827081.302715:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827081.302719:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626703488:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827081.302727:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626703616:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827081.302759:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827081.302779:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827081/real 1787827081]  req@ffff8cd0d48f52c0 x1874671626703616/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827106 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827081.302792:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626703616/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827106 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827081.302813:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827081.302817:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827081.302820:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626703616:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827081.302850:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626703744:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827081.302894:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827081.302907:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827081/real 1787827081]  req@ffff8cd0d48f7c00 x1874671626703744/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827106 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827081.302918:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626703744/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827106 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827081.302937:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827081.302942:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827081.302945:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626703744:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827081.302952:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626703872:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827081.302978:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827081.302990:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827081/real 1787827081]  req@ffff8cd0d48f61c0 x1874671626703872/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827106 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827081.303002:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626703872/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827106 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827081.303009:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827081.303012:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827081.303015:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626703872:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827081.494192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827081.878221:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827082.518197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827082.902260:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827083.542176:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827083.926206:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827084.566206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827084.950281:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827085.590158:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827085.974198:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827086.422253:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827086.422265:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827086.422270:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827086.422280:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 972
00000100:00080000:3.0:1787827086.422292:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827086.422300:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827086.422342:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626704000/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827086.422369:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827086.422388:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827086.422389:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827086.422392:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 972
00000100:00080000:3.0:1787827086.422395:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827086.422399:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827086.422408:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827086.422417:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626704128/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827086.422438:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827086.422442:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827086.422444:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827086.422449:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 972
00000100:00080000:3.0:1787827086.422453:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827086.422461:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787827086.422464:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626704000:192.168.203.145@tcp:250:kworker.0
00000008:00100000:3.0:1787827086.422470:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827086.422494:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626704256/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827086.422506:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827086.422509:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827086.422511:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827086.422515:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 972
00000100:00080000:3.0:1787827086.422520:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827086.422524:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827086.422537:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1787827086.422546:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827086.422556:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626704384/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787827086.422610:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827086/real 1787827086]  req@ffff8cd0c8eaf0c0 x1874671626704000/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827111 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827086.422625:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626704000/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827111 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827086.422633:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827086.422635:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827086.422638:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626704000:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827086.422643:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626704128:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827086.422676:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827086.422694:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827086/real 1787827086]  req@ffff8cd0c8eae940 x1874671626704128/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827111 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827086.422742:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626704128/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827111 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827086.422752:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827086.422755:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827086.422758:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626704128:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787827086.422786:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626704256:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787827086.422808:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827086.422818:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827086/real 1787827086]  req@ffff8cd0c8eada40 x1874671626704256/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827111 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827086.422830:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626704256/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827111 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827086.422846:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827086.422850:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827086.422853:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626704256:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787827086.422868:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626704384:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827086.422890:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827086.422899:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827086/real 1787827086]  req@ffff8cd0c8eaf840 x1874671626704384/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827111 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827086.422915:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626704384/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827111 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827086.422921:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827086.422923:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827086.422924:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626704384:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827086.614239:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827086.998283:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827087.638210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827088.022267:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827088.662191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827089.046271:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827089.686254:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827090.070962:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827090.710914:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827091.094299:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827091.542163:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827091.542171:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827091.542175:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827091.542184:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 977
00000100:00080000:3.0:1787827091.542193:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827091.542199:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827091.542231:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626704512/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827091.542257:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827091.542260:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827091.542262:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827091.542265:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 977
00000100:00080000:3.0:1787827091.542268:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827091.542271:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827091.542279:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827091.542289:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626704640/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827091.542298:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827091.542301:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827091.542303:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827091.542306:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 977
00000100:00100000:0.0:1787827091.542314:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626704512:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827091.542323:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827091.542328:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827091.542333:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827091.542346:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626704768/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827091.542355:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827091.542358:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827091.542360:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827091.542364:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 977
00000800:00000100:0.0:1787827091.542369:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787827091.542371:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827091.542376:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827091.542381:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827091.542390:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626704896/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787827091.542414:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827091/real 1787827091]  req@ffff8cd0c8ead2c0 x1874671626704512/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827116 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827091.542432:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626704512/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827116 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827091.542442:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827091.542446:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827091.542450:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626704512:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827091.542456:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626704640:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827091.542478:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827091.542490:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827091/real 1787827091]  req@ffff8cd0c8eac780 x1874671626704640/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827116 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827091.542500:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626704640/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827116 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827091.542515:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827091.542519:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827091.542521:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626704640:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787827091.542557:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626704768:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787827091.542578:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827091.542589:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827091/real 1787827091]  req@ffff8cd0c8eae1c0 x1874671626704768/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827116 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827091.542601:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626704768/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827116 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827091.542608:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827091.542611:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827091.542614:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626704768:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787827091.542620:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626704896:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827091.542638:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827091.542649:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827091/real 1787827091]  req@ffff8cd0c8eac3c0 x1874671626704896/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827116 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827091.542660:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626704896/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827116 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827091.542668:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827091.542670:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827091.542677:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626704896:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827091.734296:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827092.118249:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827092.758169:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827093.142235:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827093.782203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827094.166236:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827094.806213:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827095.190280:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827095.830245:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827096.214239:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827096.662221:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827096.662229:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827096.662233:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827096.662243:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 982
00000100:00080000:1.0:1787827096.662252:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827096.662258:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827096.662297:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626705024/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827096.662321:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827096.662325:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827096.662327:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827096.662330:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 982
00000100:00080000:1.0:1787827096.662332:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827096.662336:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827096.662344:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827096.662355:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626705152/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827096.662366:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1787827096.662370:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626705024:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787827096.662370:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827096.662373:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827096.662391:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 982
00000100:00080000:1.0:1787827096.662394:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827096.662397:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827096.662403:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827096.662414:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626705280/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827096.662422:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827096.662425:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827096.662427:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827096.662431:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 982
00000100:00080000:1.0:1787827096.662434:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827096.662437:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:0.0:1787827096.662442:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:1.0:1787827096.662451:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827096.662463:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626705408/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787827096.662487:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827096/real 1787827096]  req@ffff8cd0d48f70c0 x1874671626705024/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827121 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827096.662505:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626705024/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827121 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827096.662514:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827096.662518:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827096.662522:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626705024:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827096.662541:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626705152:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827096.662562:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827096.662572:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827096/real 1787827096]  req@ffff8cd0d48f43c0 x1874671626705152/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827121 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827096.662584:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626705152/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827121 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827096.662591:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827096.662594:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827096.662597:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626705152:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787827096.662611:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626705280:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787827096.662631:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827096.662641:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827096/real 1787827096]  req@ffff8cd0d48f5a40 x1874671626705280/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827121 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827096.662651:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626705280/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827121 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827096.662658:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827096.662661:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827096.662664:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626705280:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787827096.662669:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626705408:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827096.662700:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827096.662710:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827096/real 1787827096]  req@ffff8cd0d48f7840 x1874671626705408/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827121 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827096.662720:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626705408/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827121 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827096.662728:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827096.662730:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827096.662733:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626705408:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827096.854234:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827097.238256:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827097.878199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827098.262216:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827098.902195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827099.286311:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827099.926345:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827100.310234:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827100.950248:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827101.334319:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827101.782292:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827101.782308:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827101.782313:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827101.782325:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 987
00000100:00080000:3.0:1787827101.782339:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827101.782347:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827101.782401:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626705536/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827101.782430:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827101.782450:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827101.782452:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827101.782455:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 987
00000100:00080000:3.0:1787827101.782458:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827101.782461:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827101.782471:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827101.782481:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626705664/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827101.782488:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827101.782491:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827101.782492:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827101.782494:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 987
00000100:00080000:3.0:1787827101.782496:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827101.782501:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787827101.782502:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626705536:192.168.203.145@tcp:250:kworker.0
00000008:00100000:3.0:1787827101.782504:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827101.782514:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626705792/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827101.782524:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827101.782527:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827101.782528:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827101.782530:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 987
00000100:00080000:3.0:1787827101.782531:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827101.782533:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827101.782543:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827101.782554:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626705920/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787827101.782599:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827101.782641:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827101/real 1787827101]  req@ffff8cd0c8eacf00 x1874671626705536/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827101.782656:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626705536/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827101.782667:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827101.782671:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827101.782674:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626705536:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827101.782682:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626705664:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827101.782706:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827101.782718:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827101/real 1787827101]  req@ffff8cd0c8eac000 x1874671626705664/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827101.782726:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626705664/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827101.782751:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827101.782753:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827101.782756:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626705664:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787827101.782778:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626705792:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787827101.782795:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827101.782806:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827101/real 1787827101]  req@ffff8cd0c8eac3c0 x1874671626705792/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827101.782832:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626705792/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827101.782840:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827101.782843:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827101.782845:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626705792:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787827101.782851:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626705920:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827101.782872:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827101.782884:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827101/real 1787827101]  req@ffff8cd0c8eae1c0 x1874671626705920/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827101.782894:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626705920/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827101.782901:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827101.782904:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827101.782907:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626705920:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827101.974214:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827102.358231:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827102.998194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827103.382253:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827104.022199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827104.406239:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827105.046144:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827105.430308:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827106.070183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827106.454236:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827106.902207:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827106.902231:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827106.902235:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827106.902244:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 992
00000100:00080000:1.0:1787827106.902262:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827106.902269:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827106.902302:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626706048/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827106.902325:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827106.902328:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827106.902330:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827106.902334:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 992
00000100:00080000:1.0:1787827106.902337:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827106.902341:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827106.902356:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827106.902368:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626706176/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827106.902379:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1787827106.902380:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626706048:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787827106.902385:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827106.902389:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827106.902393:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 992
00000100:00080000:1.0:1787827106.902396:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827106.902399:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827106.902403:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827106.902422:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626706304/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827106.902439:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827106.902442:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827106.902444:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827106.902447:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 992
00000100:00080000:1.0:1787827106.902452:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827106.902458:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:0.0:1787827106.902459:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:1.0:1787827106.902465:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827106.902478:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626706432/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787827106.902504:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827106/real 1787827106]  req@ffff8cd0d48f4000 x1874671626706048/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827106.902553:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626706048/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827106.902564:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827106.902568:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827106.902572:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626706048:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827106.902594:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626706176:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827106.902629:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827106.902643:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827106/real 1787827106]  req@ffff8cd0d48f6580 x1874671626706176/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827106.902656:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626706176/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827106.902664:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827106.902667:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827106.902670:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626706176:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787827106.902676:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626706304:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787827106.902714:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827106.902726:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827106/real 1787827106]  req@ffff8cd0d48f7480 x1874671626706304/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827106.902738:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626706304/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827106.902757:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827106.902760:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827106.902764:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626706304:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787827106.902769:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626706432:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827106.902802:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827106.902815:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827106/real 1787827106]  req@ffff8cd0d48f4b40 x1874671626706432/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827106.902825:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626706432/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827106.902833:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827106.902836:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827106.902838:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626706432:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827107.094212:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827107.478820:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827108.118198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827108.502214:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827109.142394:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827109.526270:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827110.166188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827110.550206:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827111.190309:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827111.574300:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827112.022255:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827112.022264:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827112.022268:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827112.022278:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 997
00000100:00080000:3.0:1787827112.022288:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827112.022295:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827112.022329:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626706560/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827112.022355:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827112.022358:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827112.022360:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827112.022363:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 997
00000100:00080000:3.0:1787827112.022366:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827112.022369:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827112.022376:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827112.022386:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626706688/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827112.022393:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827112.022395:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827112.022397:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827112.022400:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 997
00000100:00100000:0.0:1787827112.022407:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626706560:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827112.022419:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827112.022424:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827112.022430:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827112.022444:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626706816/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827112.022453:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827112.022457:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827112.022459:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827112.022463:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 997
00000100:00080000:3.0:1787827112.022466:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827112.022470:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827112.022475:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1787827112.022477:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827112.022494:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626706944/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787827112.022520:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827112/real 1787827112]  req@ffff8cd0c8eac780 x1874671626706560/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827137 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827112.022539:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626706560/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827137 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827112.022548:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827112.022559:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827112.022563:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626706560:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827112.022570:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626706688:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827112.022596:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827112.022608:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827112/real 1787827112]  req@ffff8cd0c8ead2c0 x1874671626706688/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827137 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827112.022620:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626706688/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827137 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827112.022628:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827112.022630:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827112.022634:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626706688:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787827112.022655:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626706816:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787827112.022688:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827112.022699:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827112/real 1787827112]  req@ffff8cd0c8eaf840 x1874671626706816/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827137 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827112.022710:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626706816/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827137 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827112.022718:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827112.022721:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827112.022724:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626706816:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787827112.022730:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626706944:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827112.022758:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827112.022769:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827112/real 1787827112]  req@ffff8cd0c8eada40 x1874671626706944/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827137 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827112.022781:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626706944/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827137 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827112.022788:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827112.022792:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827112.022795:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626706944:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827112.214204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827112.598233:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827113.238228:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827113.622380:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827114.262254:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827114.646297:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827115.286207:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827115.670680:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827116.310206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827116.694373:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827117.142260:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827117.142270:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827117.142274:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827117.142283:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1002
00000100:00080000:3.0:1787827117.142293:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827117.142299:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827117.142334:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626707072/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827117.142374:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827117.142391:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827117.142393:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827117.142399:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1002
00000100:00080000:3.0:1787827117.142403:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827117.142406:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827117.142415:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827117.142421:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626707072:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827117.142427:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626707200/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827117.142435:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827117.142437:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827117.142439:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827117.142441:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1002
00000100:00080000:3.0:1787827117.142443:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827117.142445:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827117.142448:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827117.142464:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626707328/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827117.142470:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827117.142472:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827117.142473:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827117.142475:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1002
00000100:00080000:3.0:1787827117.142477:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827117.142480:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827117.142489:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827117.142500:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626707456/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787827117.142750:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626707200:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787827117.142983:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626707328:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787827117.143003:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626707456:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827117.335703:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827117.718329:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827118.358330:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827118.742233:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827119.382197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827119.766256:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:1.0:1787827120.214335:0:2345:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:1.0:1787827120.214368:0:2345:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1787827120.214381:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827120.214414:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827120.214431:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827120.214436:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:0.0:1787827120.215615:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827117/real 1787827120]  req@ffff8cd0c8eaf0c0 x1874671626707200/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827142 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827120.215642:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626707200/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827142 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827120.215655:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827120.215669:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827120.215672:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626707200:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787827120.215678:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827117/real 1787827120]  req@ffff8cd0c8eada40 x1874671626707328/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827142 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827120.215689:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626707328/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827142 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827120.215695:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827120.215698:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827120.215701:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626707328:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787827120.215706:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827117/real 1787827120]  req@ffff8cd0c8eaf840 x1874671626707456/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827142 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827120.215715:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626707456/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827142 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827120.215720:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827120.215723:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827120.215726:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626707456:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787827120.215730:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827117/real 1787827120]  req@ffff8cd0c8eae940 x1874671626707072/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827142 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827120.215739:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626707072/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827142 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827120.215745:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827120.215752:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827120.215754:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626707072:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787827120.406258:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827120.790270:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827121.430202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827121.814818:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827122.262207:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827122.262222:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827122.262226:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827122.262236:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1008
00000100:00080000:3.0:1787827122.262245:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827122.262253:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827122.262348:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626707584/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827122.262377:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827122.262381:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827122.262383:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827122.262386:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1008
00000100:00080000:3.0:1787827122.262390:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827122.262394:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827122.262402:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827122.262422:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626707712/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827122.262433:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827122.262436:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827122.262438:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827122.262441:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1008
00000100:00100000:0.0:1787827122.262459:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626707584:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827122.262461:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827122.262470:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827122.262476:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827122.262489:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626707840/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827122.262499:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827122.262502:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827122.262504:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827122.262508:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1008
00000100:00080000:3.0:1787827122.262512:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827122.262516:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:0.0:1787827122.262521:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:3.0:1787827122.262523:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827122.262535:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626707968/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787827122.262560:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827122/real 1787827122]  req@ffff8cd0c8ead2c0 x1874671626707584/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827147 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827122.262578:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626707584/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827147 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827122.262915:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827122.262921:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827122.262925:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626707584:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827122.262934:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626707712:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827122.262961:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827122.262976:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827122/real 1787827122]  req@ffff8cd0c8eac780 x1874671626707712/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827147 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827122.262988:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626707712/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827147 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827122.262996:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827122.262999:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827122.263002:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626707712:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787827122.263026:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626707840:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787827122.263059:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827122.263070:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827122/real 1787827122]  req@ffff8cd0c8eae1c0 x1874671626707840/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827147 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827122.263080:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626707840/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827147 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827122.263116:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827122.263119:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827122.263122:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626707840:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787827122.263128:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626707968:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827122.263146:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827122.263157:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827122/real 1787827122]  req@ffff8cd0c8eac3c0 x1874671626707968/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827147 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827122.263167:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626707968/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827147 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827122.263174:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827122.263177:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827122.263180:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626707968:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827122.454165:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827122.838253:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827123.478194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827123.862767:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827124.502197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827124.886277:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827125.526656:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827125.910247:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827126.550265:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827126.934273:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827127.382207:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827127.382216:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827127.382220:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827127.382234:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1013
00000100:00080000:2.0:1787827127.382244:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827127.382249:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827127.382288:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daad00 x1874671626708096/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827127.382312:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827127.382314:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827127.382315:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827127.382318:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1013
00000100:00080000:2.0:1787827127.382321:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827127.382323:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827127.382331:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827127.382342:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626708224/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827127.382349:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1787827127.382353:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626708096:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787827127.382353:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827127.382357:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827127.382362:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1013
00000100:00080000:2.0:1787827127.382364:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827127.382366:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827127.382388:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827127.382397:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626708352/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827127.382403:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1787827127.382407:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787827127.382407:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827127.382409:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827127.382412:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1013
00000100:00080000:2.0:1787827127.382415:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827127.382419:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827127.382422:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827127.382428:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626708480/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787827127.382459:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827127/real 1787827127]  req@ffff8cd0c3daad00 x1874671626708096/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827152 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827127.382476:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daad00 x1874671626708096/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827152 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827127.382483:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827127.382486:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827127.382488:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626708096:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827127.382505:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626708224:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827127.382521:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827127.382529:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827127/real 1787827127]  req@ffff8cd0c3daa580 x1874671626708224/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827152 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827127.382537:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626708224/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827152 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827127.382542:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827127.382545:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827127.382548:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626708224:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787827127.382565:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626708352:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787827127.382594:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827127.382606:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827127/real 1787827127]  req@ffff8cd0c3da92c0 x1874671626708352/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827152 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827127.382613:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626708352/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827152 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827127.382626:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827127.382628:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827127.382630:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626708352:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787827127.382634:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626708480:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827127.382648:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827127.382661:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827127/real 1787827127]  req@ffff8cd0c3da9e00 x1874671626708480/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827152 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827127.382667:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626708480/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827152 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827127.382672:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827127.382674:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827127.382676:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626708480:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827127.574195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827127.958283:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827128.598203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827128.982329:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827129.622167:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827130.006248:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827130.646193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827131.030233:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827131.670196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827132.054215:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827132.502251:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827132.502261:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827132.502265:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827132.502275:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1018
00000100:00080000:3.0:1787827132.502285:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827132.502293:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827132.502327:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626708608/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827132.502359:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827132.502378:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827132.502380:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827132.502385:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1018
00000100:00080000:3.0:1787827132.502389:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827132.502392:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827132.502403:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827132.502416:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626708736/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827132.502426:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827132.502429:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827132.502431:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827132.502434:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1018
00000100:00080000:3.0:1787827132.502437:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827132.502440:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827132.502444:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827132.502455:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626708864/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827132.502464:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827132.502467:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827132.502468:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827132.502471:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1018
00000100:00080000:3.0:1787827132.502474:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827132.502478:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827132.502489:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827132.502500:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626708992/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787827132.502603:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626708608:192.168.203.145@tcp:250:kworker.0
00000800:00000100:0.0:1787827132.502671:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827132.502709:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827132/real 1787827132]  req@ffff8cd0c8eac000 x1874671626708608/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827157 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827132.502726:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626708608/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827157 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827132.502744:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827132.502748:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827132.502751:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626708608:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827132.502768:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626708736:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827132.502796:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827132.502809:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827132/real 1787827132]  req@ffff8cd0c8eacf00 x1874671626708736/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827157 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827132.502820:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626708736/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827157 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827132.502828:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827132.502831:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827132.502834:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626708736:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787827132.502840:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626708864:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787827132.502859:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827132.502870:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827132/real 1787827132]  req@ffff8cd0c8eac3c0 x1874671626708864/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827157 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827132.502881:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626708864/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827157 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827132.502888:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827132.502891:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827132.502894:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626708864:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787827132.502906:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626708992:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827132.502928:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827132.502940:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827132/real 1787827132]  req@ffff8cd0c8eae1c0 x1874671626708992/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827157 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827132.502950:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626708992/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827157 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827132.502959:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827132.502962:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827132.502964:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626708992:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827132.694204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827133.078257:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827133.718206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827134.102227:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827134.742195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827135.126254:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827135.766222:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827136.150281:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827136.790224:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827137.174241:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827137.622364:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827137.622373:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827137.622377:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827137.622386:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1023
00000100:00080000:2.0:1787827137.622397:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827137.622404:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827137.622436:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626709120/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827137.622464:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827137.622466:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827137.622468:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827137.622470:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1023
00000100:00080000:2.0:1787827137.622495:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827137.622499:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827137.622507:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827137.622513:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626709120:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827137.622524:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626709248/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827137.622535:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827137.622538:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827137.622540:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827137.622543:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1023
00000100:00080000:2.0:1787827137.622547:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827137.622551:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827137.622556:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827137.622568:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626709376/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787827137.622574:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787827137.622585:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827137.622587:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827137.622589:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827137.622593:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1023
00000100:00080000:2.0:1787827137.622596:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827137.622599:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827137.622604:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827137.622616:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626709504/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787827137.622619:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827137/real 1787827137]  req@ffff8cd0c3da8000 x1874671626709120/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827162 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827137.622645:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626709120/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827162 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827137.622657:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827137.622661:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827137.622666:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626709120:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827137.622703:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626709248:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827137.622727:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827137.622740:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827137/real 1787827137]  req@ffff8cd0c3dabc00 x1874671626709248/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827162 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827137.622753:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626709248/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827162 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827137.622761:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827137.622764:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827137.622767:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626709248:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787827137.622773:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626709376:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787827137.622792:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827137.622803:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827137/real 1787827137]  req@ffff8cd0c3daa1c0 x1874671626709376/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827162 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827137.622814:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626709376/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827162 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827137.622821:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827137.622824:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827137.622827:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626709376:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787827137.622833:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626709504:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827137.622852:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827137.622864:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827137/real 1787827137]  req@ffff8cd0c3dab840 x1874671626709504/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827162 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827137.622884:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626709504/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827162 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827137.622892:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827137.622895:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827137.622898:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626709504:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827137.814212:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827138.198226:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827138.838196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827139.222276:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827139.862621:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827140.246667:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827140.886183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827141.270257:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827141.910209:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827142.294237:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827142.742212:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827142.742222:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827142.742226:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827142.742243:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1028
00000100:00080000:1.0:1787827142.742253:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827142.742261:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827142.742302:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626709632/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827142.742338:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827142.742341:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827142.742343:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827142.742347:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1028
00000100:00080000:1.0:1787827142.742351:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827142.742356:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827142.742365:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827142.742372:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626709632:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787827142.742379:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626709760/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827142.742390:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827142.742394:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827142.742396:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827142.742399:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1028
00000100:00080000:1.0:1787827142.742415:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827142.742419:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:0.0:1787827142.742424:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:1.0:1787827142.742425:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827142.742439:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626709888/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827142.742460:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827142.742463:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827142.742465:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827142.742469:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1028
00000100:00100000:0.0:1787827142.742476:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827142/real 1787827142]  req@ffff8cd0d48f4b40 x1874671626709632/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827167 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787827142.742480:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827142.742488:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827142.742494:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827142.742499:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626709632/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827167 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787827142.742507:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626710016/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827142.742510:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827142.742521:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827142.742528:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626709632:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827142.742549:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626709760:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827142.742566:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827142.742575:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827142/real 1787827142]  req@ffff8cd0d48f7480 x1874671626709760/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827167 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827142.742589:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626709760/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827167 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827142.742594:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827142.742596:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827142.742598:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626709760:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787827142.742602:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626709888:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787827142.742615:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827142.742621:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827142/real 1787827142]  req@ffff8cd0d48f6580 x1874671626709888/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827167 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827142.742630:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626709888/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827167 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827142.742637:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827142.742640:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827142.742643:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626709888:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787827142.742658:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626710016:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827142.742675:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827142.742683:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827142/real 1787827142]  req@ffff8cd0d48f4000 x1874671626710016/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827167 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827142.742690:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626710016/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827167 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827142.742702:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827142.742705:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827142.742707:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626710016:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827142.934236:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827143.318209:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827143.958204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827144.342242:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827144.984240:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827145.366287:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827146.006217:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827146.390200:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827147.030189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827147.414241:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827147.862192:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827147.862198:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827147.862201:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827147.862208:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1033
00000100:00080000:3.0:1787827147.862217:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827147.862221:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827147.862249:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626710144/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827147.862273:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827147.862276:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827147.862277:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827147.862279:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1033
00000100:00080000:3.0:1787827147.862281:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827147.862283:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827147.862290:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827147.862298:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626710272/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827147.862305:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827147.862307:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827147.862309:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1787827147.862311:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626710144:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827147.862313:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1033
00000100:00080000:3.0:1787827147.862327:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827147.862330:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827147.862334:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827147.862342:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626710400/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827147.862348:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827147.862350:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827147.862351:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827147.862354:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1033
00000100:00080000:3.0:1787827147.862356:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827147.862358:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827147.862360:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1787827147.862365:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827147.862368:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626710528/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787827147.862406:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827147/real 1787827147]  req@ffff8cd0c8eac780 x1874671626710144/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827172 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827147.862438:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626710144/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827172 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827147.862447:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827147.862452:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827147.862456:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626710144:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827147.862476:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626710272:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827147.862494:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827147.862503:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827147/real 1787827147]  req@ffff8cd0c8ead2c0 x1874671626710272/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827172 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827147.862510:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626710272/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827172 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827147.862522:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827147.862524:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827147.862526:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626710272:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787827147.862530:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626710400:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787827147.862542:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827147.862548:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827147/real 1787827147]  req@ffff8cd0c8eae940 x1874671626710400/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827172 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827147.862557:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626710400/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827172 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827147.862564:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827147.862567:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827147.862570:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626710400:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787827147.862576:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626710528:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827147.862595:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827147.862606:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827147/real 1787827147]  req@ffff8cd0c8eaf840 x1874671626710528/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827172 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827147.862618:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626710528/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827172 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827147.862625:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827147.862628:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827147.862637:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626710528:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827148.054198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827148.438230:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827149.078688:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827149.462290:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827150.102198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827150.486230:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827151.126212:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827151.510228:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827152.150199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827152.534254:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827152.982188:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827152.982197:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827152.982200:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827152.982209:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1038
00000100:00080000:3.0:1787827152.982219:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827152.982225:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827152.982259:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626710656/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827152.982287:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827152.982305:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827152.982307:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827152.982311:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1038
00000100:00080000:3.0:1787827152.982315:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827152.982320:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827152.982330:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827152.982339:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626710656:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827152.982346:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626710784/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827152.982358:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827152.982361:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827152.982363:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827152.982366:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1038
00000100:00080000:3.0:1787827152.982370:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827152.982373:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827152.982393:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827152.982404:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626710912/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787827152.982413:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787827152.982417:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827152.982421:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827152.982424:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827152.982429:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1038
00000100:00080000:3.0:1787827152.982433:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827152.982436:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827152.982447:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827152.982458:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626711040/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787827152.982466:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827152/real 1787827152]  req@ffff8cd0c8eada40 x1874671626710656/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827177 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827152.982481:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626710656/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827177 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827152.982489:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827152.982492:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827152.982495:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626710656:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827152.982522:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626710784:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827152.982546:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827152.982558:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827152/real 1787827152]  req@ffff8cd0c8eaf0c0 x1874671626710784/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827177 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827152.982567:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626710784/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827177 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827152.982572:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827152.982574:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827152.982581:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626710784:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787827152.982585:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626710912:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787827152.982606:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827152.982621:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827152/real 1787827152]  req@ffff8cd0c8eaf840 x1874671626710912/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827177 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827152.982634:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626710912/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827177 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827152.982641:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827152.982644:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827152.982647:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626710912:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787827152.982653:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626711040:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827152.982672:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827152.982682:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827152/real 1787827152]  req@ffff8cd0c8eae940 x1874671626711040/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827177 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827152.982693:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626711040/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827177 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827152.982698:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827152.982700:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827152.982703:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626711040:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827153.174270:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827153.558219:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827154.198201:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827154.582230:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827155.222210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827155.606234:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827156.247052:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827156.630261:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827157.271277:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827157.654329:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827158.102274:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827158.102284:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827158.102288:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827158.102297:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1043
00000100:00080000:1.0:1787827158.102308:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827158.102315:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827158.102362:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626711168/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827158.102389:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827158.102406:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827158.102408:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827158.102412:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1043
00000100:00080000:1.0:1787827158.102416:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827158.102420:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827158.102440:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827158.102454:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626711296/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827158.102465:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827158.102468:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827158.102470:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827158.102473:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1043
00000100:00080000:1.0:1787827158.102476:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827158.102482:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787827158.102483:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626711168:192.168.203.145@tcp:250:kworker.0
00000008:00100000:1.0:1787827158.102489:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827158.102515:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626711424/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827158.102526:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827158.102529:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827158.102531:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1787827158.102534:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827158.102575:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827158/real 1787827158]  req@ffff8cd0d48f7840 x1874671626711168/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827183 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827158.102601:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626711168/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827183 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827158.102619:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827158.102623:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827158.102627:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626711168:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827158.102634:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626711296:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827158.102657:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827158.102670:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827158/real 1787827158]  req@ffff8cd0d48f5a40 x1874671626711296/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827183 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827158.102681:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626711296/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827183 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827158.102689:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827158.102693:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827158.102696:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626711296:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787827158.102720:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626711424:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787827158.102749:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827158.102761:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827158/real 1787827158]  req@ffff8cd0d48f43c0 x1874671626711424/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827183 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827158.102772:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626711424/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827183 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827158.102780:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827158.102783:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827158.102786:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626711424:192.168.203.145@tcp:8:kworker.0
00000100:00080000:1.0:1787827158.102814:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1043
00000100:00080000:1.0:1787827158.102819:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827158.102823:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827158.102835:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827158.102847:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626711552/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787827158.102916:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626711552:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827158.102941:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827158.102952:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827158/real 1787827158]  req@ffff8cd0d48f70c0 x1874671626711552/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827183 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827158.102963:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626711552/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827183 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827158.102971:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827158.102975:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827158.102978:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626711552:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827158.294242:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827158.678234:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827159.318276:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827159.702211:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827160.342238:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827160.726281:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827161.366642:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827161.750393:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827162.390193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827162.774229:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827163.222213:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827163.222221:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827163.222225:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827163.222234:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1049
00000100:00080000:3.0:1787827163.222242:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827163.222247:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827163.222276:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626711680/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827163.222296:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827163.222298:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827163.222299:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827163.222301:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1049
00000100:00080000:3.0:1787827163.222318:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827163.222321:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827163.222328:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827163.222337:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626711808/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787827163.222347:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626711680:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827163.222358:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827163.222361:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827163.222362:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827163.222365:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1049
00000100:00080000:3.0:1787827163.222368:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827163.222371:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827163.222374:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827163.222383:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626711936/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827163.222389:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827163.222392:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827163.222393:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827163.222395:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1049
00000100:00080000:3.0:1787827163.222397:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827163.222400:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827163.222402:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827163.222408:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626712064/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787827163.222414:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827163.222451:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827163/real 1787827163]  req@ffff8cd0c8ead2c0 x1874671626711680/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827188 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827163.222468:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626711680/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827188 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827163.222478:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827163.222482:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827163.222486:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626711680:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827163.222504:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626711808:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827163.222522:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827163.222531:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827163/real 1787827163]  req@ffff8cd0c8eac780 x1874671626711808/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827188 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827163.222542:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626711808/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827188 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827163.222550:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827163.222553:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827163.222556:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626711808:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787827163.222561:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626711936:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787827163.222574:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827163.222582:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827163/real 1787827163]  req@ffff8cd0c8eae1c0 x1874671626711936/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827188 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827163.222606:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626711936/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827188 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827163.222614:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827163.222617:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827163.222620:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626711936:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787827163.222624:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626712064:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827163.222642:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827163.222652:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827163/real 1787827163]  req@ffff8cd0c8eac3c0 x1874671626712064/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827188 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827163.222663:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626712064/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827188 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827163.222671:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827163.222674:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827163.222677:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626712064:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827163.414185:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827163.798240:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827164.438170:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827164.822201:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827165.462192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827165.846226:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827166.486214:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827166.870212:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827167.510175:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827167.894298:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787827168.342212:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827168.342221:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787827168.342226:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827168.342251:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1054
00000100:00080000:0.0:1787827168.342261:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827168.342279:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787827168.342310:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626712192/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827168.342337:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827168.342340:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827168.342342:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827168.342346:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1054
00000100:00080000:0.0:1787827168.342350:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827168.342354:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827168.342365:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827168.342367:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626712192:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827168.342379:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626712320/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827168.342391:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827168.342395:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787827168.342397:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827168.342401:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1054
00000100:00080000:0.0:1787827168.342405:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827168.342410:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827168.342415:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787827168.342422:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827168.342427:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626712448/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827168.342449:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827168.342453:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827168.342455:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1787827168.342457:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827168/real 1787827168]  req@ffff8cd0c8e9c000 x1874671626712192/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827193 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827168.342462:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1054
00000100:00080000:0.0:1787827168.342468:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827168.342476:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827168.342480:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626712192/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827193 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:0.0:1787827168.342483:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00080000:3.0:1787827168.342492:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827168.342496:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827168.342502:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626712192:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827168.342503:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626712576/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827168.342521:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626712320:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827168.342539:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827168.342547:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827168/real 1787827168]  req@ffff8cd0c8e9f840 x1874671626712320/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827193 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827168.342555:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626712320/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827193 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827168.342566:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827168.342568:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827168.342570:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626712320:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827168.342574:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626712448:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827168.342586:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827168.342601:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827168/real 1787827168]  req@ffff8cd0c8e9ed00 x1874671626712448/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827193 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827168.342626:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626712448/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827193 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827168.342634:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827168.342637:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827168.342640:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626712448:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827168.342645:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626712576:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827168.342664:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827168.342674:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827168/real 1787827168]  req@ffff8cd0c8e9d680 x1874671626712576/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827193 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827168.342684:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626712576/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827193 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827168.342691:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827168.342694:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827168.342697:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626712576:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827168.534540:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827168.918210:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827169.558314:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827169.942238:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827170.582153:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827170.966163:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827171.606172:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827171.990211:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827172.630164:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827173.014250:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827173.462265:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827173.462275:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827173.462279:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827173.462304:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1059
00000100:00080000:3.0:1787827173.462314:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827173.462321:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827173.462354:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626712704/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787827173.462416:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626712704:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827173.462457:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827173.462461:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827173.462463:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827173.462467:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1059
00000100:00080000:3.0:1787827173.462471:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827173.462474:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:0.0:1787827173.462477:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:3.0:1787827173.462483:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827173.462495:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626712832/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827173.462504:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827173.462507:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827173.462508:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827173.462512:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1059
00000100:00100000:0.0:1787827173.462514:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827173/real 1787827173]  req@ffff8cd0c8eacf00 x1874671626712704/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827198 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827173.462514:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827173.462518:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827173.462522:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827173.462529:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626712704/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827198 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827173.462530:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626712960/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827173.462540:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827173.462544:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827173.462548:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626712704:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827173.462550:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827173.462553:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827173.462554:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827173.462558:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1059
00000100:00080000:3.0:1787827173.462561:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827173.462564:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827173.462568:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827173.462576:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626712832:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827173.462577:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626713088/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787827173.462597:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827173.462609:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827173/real 1787827173]  req@ffff8cd0c8eac000 x1874671626712832/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827198 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827173.462620:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626712832/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827198 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827173.462628:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827173.462632:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827173.462635:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626712832:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787827173.462640:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626712960:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787827173.462655:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827173.462672:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827173/real 1787827173]  req@ffff8cd0c8eac3c0 x1874671626712960/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827198 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827173.462682:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626712960/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827198 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827173.462690:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827173.462702:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827173.462705:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626712960:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787827173.462719:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626713088:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827173.462735:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827173.462742:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827173/real 1787827173]  req@ffff8cd0c8eae1c0 x1874671626713088/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827198 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827173.462755:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626713088/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827198 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827173.462762:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827173.462765:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827173.462769:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626713088:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827173.654154:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827174.038211:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827174.678188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827175.062238:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827175.702189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827176.086246:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827176.726159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827177.110263:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827177.750240:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827178.134196:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827178.582242:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827178.582252:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827178.582257:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827178.582266:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1064
00000100:00080000:2.0:1787827178.582278:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827178.582284:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827178.582327:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626713216/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827178.582358:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827178.582361:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827178.582364:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827178.582367:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1064
00000100:00080000:2.0:1787827178.582371:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827178.582375:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827178.582387:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827178.582403:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626713216:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827178.582404:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626713344/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827178.582419:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827178.582423:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827178.582425:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827178.582429:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1064
00000100:00080000:2.0:1787827178.582433:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827178.582437:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827178.582460:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827178.582471:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626713472/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787827178.582476:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787827178.582484:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827178.582487:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827178.582488:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827178.582492:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1064
00000100:00080000:2.0:1787827178.582496:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827178.582499:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827178.582502:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827178.582510:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626713600/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787827178.582525:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827178/real 1787827178]  req@ffff8cd0c3da8780 x1874671626713216/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827203 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827178.582539:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626713216/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827203 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827178.582548:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827178.582557:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827178.582560:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626713216:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827178.582581:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626713344:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827178.582607:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827178.582621:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827178/real 1787827178]  req@ffff8cd0c3da9a40 x1874671626713344/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827203 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827178.582640:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626713344/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827203 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827178.582647:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827178.582650:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827178.582653:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626713344:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787827178.582658:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626713472:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787827178.582680:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827178.582691:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827178/real 1787827178]  req@ffff8cd0c3dab0c0 x1874671626713472/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827203 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827178.582703:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626713472/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827203 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827178.582709:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827178.582711:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827178.582713:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626713472:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787827178.582725:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626713600:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827178.582744:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827178.582755:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827178/real 1787827178]  req@ffff8cd0c3daa940 x1874671626713600/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827203 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827178.582766:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626713600/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827203 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827178.582784:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827178.582787:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827178.582790:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626713600:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827178.774225:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827179.158222:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827179.798174:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827180.182203:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827180.822194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827181.206258:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827181.846142:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827182.230240:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827182.870184:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827183.254269:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827183.702214:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827183.702225:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827183.702229:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827183.702238:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1069
00000100:00080000:3.0:1787827183.702249:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827183.702255:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827183.702320:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626713728/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827183.702346:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827183.702348:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827183.702349:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827183.702352:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1069
00000100:00080000:3.0:1787827183.702355:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827183.702358:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827183.702368:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827183.702379:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626713856/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787827183.702386:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626713728:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827183.702388:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827183.702394:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827183.702396:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827183.702398:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1069
00000100:00080000:3.0:1787827183.702400:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827183.702402:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827183.702405:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827183.702413:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626713984/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827183.702419:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827183.702422:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827183.702423:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827183.702431:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1069
00000100:00080000:3.0:1787827183.702434:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827183.702436:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827183.702440:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827183.702447:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626714112/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787827183.702497:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626713856:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787827183.702520:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626713984:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787827183.702535:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626714112:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827183.894185:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827184.278213:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827184.918222:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827185.302255:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827185.942198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827186.326231:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:3.0:1787827186.774385:0:2344:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:3.0:1787827186.774438:0:2344:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:3.0:1787827186.774455:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:3.0:1787827186.774491:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:3.0:1787827186.774500:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:3.0:1787827186.774509:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:0.0:1787827186.774533:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827183/real 1787827186]  req@ffff8cd0c8ead2c0 x1874671626713856/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827208 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827186.774557:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626713856/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827208 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827186.774567:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827186.774572:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827186.774575:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626713856:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787827186.774596:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827183/real 1787827186]  req@ffff8cd0c8eae940 x1874671626713984/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827208 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827186.774616:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626713984/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827208 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827186.774623:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827186.774626:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827186.774630:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626713984:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787827186.774636:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827183/real 1787827186]  req@ffff8cd0c8eaf840 x1874671626714112/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827208 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827186.774646:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626714112/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827208 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827186.774653:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827186.774657:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827186.774659:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626714112:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787827186.774666:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827183/real 1787827186]  req@ffff8cd0c8eac780 x1874671626713728/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827208 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827186.774676:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626713728/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827208 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827186.774683:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827186.774686:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827186.774689:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626713728:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827186.966192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827187.350210:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827187.990193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827188.374262:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827188.822218:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827188.822227:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827188.822231:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827188.822242:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1074
00000100:00080000:2.0:1787827188.822254:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827188.822261:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827188.822296:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626714240/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827188.822327:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827188.822331:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827188.822333:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827188.822337:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1074
00000100:00080000:2.0:1787827188.822355:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827188.822361:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827188.822371:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827188.822383:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626714240:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827188.822396:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626714368/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827188.822410:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827188.822414:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827188.822415:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827188.822419:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1074
00000100:00080000:2.0:1787827188.822425:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827188.822429:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827188.822436:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1787827188.822445:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827188.822458:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626714496/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827188.822468:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827188.822472:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827188.822475:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827188.822479:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1074
00000100:00080000:2.0:1787827188.822484:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1787827188.822487:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827188/real 1787827188]  req@ffff8cd0c3da83c0 x1874671626714240/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827188.822492:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827188.822501:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827188.822516:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626714240/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827188.822518:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab480 x1874671626714624/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827188.822543:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827188.822550:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827188.822560:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626714240:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827188.822588:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626714368:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827188.822612:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827188.822625:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827188/real 1787827188]  req@ffff8cd0c3da9680 x1874671626714368/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827188.822638:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626714368/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827188.822647:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827188.822651:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827188.822654:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626714368:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787827188.822660:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626714496:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787827188.822689:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827188.822707:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827188/real 1787827188]  req@ffff8cd0c3da8f00 x1874671626714496/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827188.822720:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626714496/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827188.822729:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827188.822733:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827188.822736:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626714496:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787827188.822743:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626714624:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827188.822761:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827188.822771:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827188/real 1787827188]  req@ffff8cd0c3dab480 x1874671626714624/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827188.822783:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab480 x1874671626714624/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827188.822792:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827188.822794:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827188.822797:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626714624:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827189.014194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827189.398231:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827190.038191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827190.422291:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827191.062216:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827191.446207:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827192.086217:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827192.470249:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827193.110232:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827193.494290:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827193.942251:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827193.942259:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827193.942263:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827193.942271:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1079
00000100:00080000:3.0:1787827193.942282:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827193.942306:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827193.942339:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626714752/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827193.942368:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827193.942371:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827193.942372:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827193.942375:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1079
00000100:00080000:3.0:1787827193.942378:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827193.942381:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827193.942389:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827193.942407:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626714880/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827193.942419:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827193.942421:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827193.942423:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827193.942426:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1079
00000100:00080000:3.0:1787827193.942429:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827193.942432:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787827193.942453:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626714752:192.168.203.145@tcp:250:kworker.0
00000008:00100000:3.0:1787827193.942454:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827193.942468:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626715008/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827193.942477:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827193.942479:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827193.942486:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827193.942490:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1079
00000100:00080000:3.0:1787827193.942494:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827193.942497:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827193.942502:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827193.942511:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626715136/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787827193.942521:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827193.942558:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827193/real 1787827193]  req@ffff8cd0c8eaf0c0 x1874671626714752/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827218 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827193.942571:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626714752/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827218 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827193.942579:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827193.942581:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827193.942584:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626714752:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827193.942589:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626714880:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827193.942604:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827193.942611:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827193/real 1787827193]  req@ffff8cd0c8eada40 x1874671626714880/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827218 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827193.942619:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626714880/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827218 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827193.942623:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827193.942625:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827193.942627:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626714880:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787827193.942647:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626715008:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787827193.942658:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827193.942665:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827193/real 1787827193]  req@ffff8cd0c8eac780 x1874671626715008/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827218 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827193.942683:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626715008/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827218 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827193.942688:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827193.942690:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827193.942692:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626715008:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787827193.942696:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626715136:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827193.942708:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827193.942714:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827193/real 1787827193]  req@ffff8cd0c8eaf840 x1874671626715136/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827218 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827193.942721:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626715136/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827218 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827193.942726:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827193.942728:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827193.942730:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626715136:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827194.134205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827194.518208:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827195.158197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827195.542239:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827196.182194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827196.566217:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827197.206179:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827197.590208:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827198.230180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827198.614285:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787827199.062255:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827199.062271:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787827199.062277:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827199.062310:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1084
00000100:00080000:0.0:1787827199.062324:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827199.062331:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787827199.062371:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626715264/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827199.062400:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827199.062408:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827199.062409:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827199.062413:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1084
00000100:00080000:0.0:1787827199.062416:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827199.062419:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827199.062430:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827199.062448:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626715392/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827199.062456:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626715264:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787827199.062463:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827199.062469:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787827199.062471:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827199.062476:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1084
00000100:00080000:0.0:1787827199.062480:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827199.062485:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827199.062493:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827199.062510:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626715520/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827199.062536:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827199.062540:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827199.062543:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:3.0:1787827199.062546:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787827199.062551:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1084
00000100:00080000:0.0:1787827199.062558:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827199.062563:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827199.062570:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827199.062586:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626715648/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827199.062609:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827199/real 1787827199]  req@ffff8cd0c8e9cf00 x1874671626715264/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827224 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827199.062633:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626715264/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827224 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827199.062645:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827199.062650:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827199.062655:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626715264:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827199.062686:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626715392:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827199.062712:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827199.062726:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827199/real 1787827199]  req@ffff8cd0c8e9e940 x1874671626715392/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827224 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827199.062749:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626715392/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827224 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827199.062758:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827199.062761:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827199.062765:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626715392:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827199.062773:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626715520:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827199.062798:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827199.062811:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827199/real 1787827199]  req@ffff8cd0c8e9e580 x1874671626715520/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827224 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827199.062834:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626715520/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827224 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827199.062842:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827199.062846:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827199.062850:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626715520:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827199.062856:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626715648:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827199.062878:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827199.062890:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827199/real 1787827199]  req@ffff8cd0c8e9de00 x1874671626715648/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827224 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827199.062902:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626715648/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827224 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827199.062923:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827199.062926:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827199.062930:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626715648:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827199.254226:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827199.638220:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827200.278235:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827200.662289:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827201.302182:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827201.686217:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827202.326236:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827202.710209:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827203.350139:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827203.734205:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787827204.182236:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827204.182248:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787827204.182253:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827204.182264:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1090
00000100:00080000:0.0:1787827204.182278:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827204.182284:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787827204.182330:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626715776/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827204.182359:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827204.182362:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827204.182381:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827204.182384:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1090
00000100:00080000:0.0:1787827204.182388:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827204.182391:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827204.182402:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827204.182415:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626715904/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827204.182424:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827204.182428:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787827204.182430:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827204.182438:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1090
00000100:00100000:3.0:1787827204.182440:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626715776:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787827204.182442:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827204.182451:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827204.182458:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827204.182473:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626716032/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827204.182484:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827204.182487:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827204.182489:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827204.182493:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1090
00000100:00080000:0.0:1787827204.182495:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827204.182498:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827204.182501:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827204.182530:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626716160/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787827204.182542:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827204.182596:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827204/real 1787827204]  req@ffff8cd0c8e9fc00 x1874671626715776/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827229 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827204.182625:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626715776/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827229 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827204.182637:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827204.182642:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827204.182647:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626715776:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827204.182655:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626715904:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827204.182683:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827204.182707:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827204/real 1787827204]  req@ffff8cd0c8e9de00 x1874671626715904/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827229 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827204.182722:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626715904/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827229 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827204.182739:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827204.182743:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827204.182746:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626715904:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827204.182782:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626716032:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827204.182800:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827204.182809:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827204/real 1787827204]  req@ffff8cd0c8e9e580 x1874671626716032/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827229 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827204.182817:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626716032/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827229 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827204.182823:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827204.182825:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827204.182828:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626716032:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827204.182831:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626716160:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827204.182841:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827204.182847:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827204/real 1787827204]  req@ffff8cd0c8e9e940 x1874671626716160/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827229 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827204.182854:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626716160/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827229 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827204.182860:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827204.182861:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827204.182869:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626716160:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827204.374216:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827204.758284:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827205.398337:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827205.782314:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827206.422436:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827206.806276:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827207.446210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827207.830337:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827208.470191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827208.854237:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827209.302189:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827209.302197:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827209.302201:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827209.302209:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1095
00000100:00080000:3.0:1787827209.302219:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827209.302225:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827209.302256:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626716288/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827209.302278:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827209.302281:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827209.302283:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827209.302285:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1095
00000100:00080000:3.0:1787827209.302288:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827209.302291:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827209.302298:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827209.302307:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626716416/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827209.302316:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1787827209.302323:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626716288:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827209.302333:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827209.302334:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827209.302338:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1095
00000100:00080000:3.0:1787827209.302341:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827209.302344:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827209.302348:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827209.302358:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626716544/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827209.302366:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827209.302368:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827209.302369:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827209.302372:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1095
00000100:00080000:3.0:1787827209.302375:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827209.302377:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827209.302380:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827209.302386:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626716672/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787827209.302396:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827209.302438:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827209/real 1787827209]  req@ffff8cd0c8eae940 x1874671626716288/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827234 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827209.302455:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626716288/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827234 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827209.302466:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827209.302470:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827209.302474:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626716288:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827209.302498:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626716416:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827209.302519:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827209.302539:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827209/real 1787827209]  req@ffff8cd0c8ead2c0 x1874671626716416/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827234 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827209.302553:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626716416/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827234 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827209.302562:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827209.302565:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827209.302576:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626716416:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787827209.302582:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626716544:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787827209.302616:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827209.302628:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827209/real 1787827209]  req@ffff8cd0c8eae1c0 x1874671626716544/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827234 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827209.302641:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626716544/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827234 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827209.302650:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827209.302653:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827209.302657:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626716544:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787827209.302664:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626716672:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827209.302684:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827209.302697:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827209/real 1787827209]  req@ffff8cd0c8eac3c0 x1874671626716672/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827234 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827209.302711:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626716672/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827234 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827209.302719:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827209.302723:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827209.302726:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626716672:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827209.494155:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827209.878249:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827210.518149:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827210.902208:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827211.542190:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827211.926212:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827212.566234:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827212.950192:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827213.590184:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827213.974209:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827214.422254:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827214.422268:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827214.422273:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827214.422286:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1100
00000100:00080000:2.0:1787827214.422301:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827214.422309:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827214.422358:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8b40 x1874671626716800/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827214.422392:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827214.422396:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827214.422398:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827214.422401:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1100
00000100:00080000:2.0:1787827214.422405:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827214.422409:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827214.422422:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827214.422439:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab480 x1874671626716928/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827214.422453:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1787827214.422455:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626716800:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787827214.422465:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827214.422469:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827214.422475:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1100
00000100:00080000:2.0:1787827214.422478:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827214.422483:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827214.422507:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827214.422522:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626717056/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787827214.422534:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787827214.422535:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827214.422542:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827214.422545:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827214.422552:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1100
00000100:00080000:2.0:1787827214.422558:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827214.422563:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827214.422568:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827214.422580:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827214/real 1787827214]  req@ffff8cd0c3da8b40 x1874671626716800/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827239 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827214.422585:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626717184/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787827214.422633:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8b40 x1874671626716800/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827239 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827214.422644:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827214.422648:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827214.422655:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626716800:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827214.422681:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626716928:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827214.422699:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827214.422709:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827214/real 1787827214]  req@ffff8cd0c3dab480 x1874671626716928/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827239 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827214.422717:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab480 x1874671626716928/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827239 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827214.422723:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827214.422725:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827214.422728:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626716928:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787827214.422732:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626717056:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787827214.422744:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827214.422750:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827214/real 1787827214]  req@ffff8cd0c3da8f00 x1874671626717056/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827239 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827214.422770:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626717056/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827239 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827214.422776:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827214.422778:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827214.422780:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626717056:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787827214.422784:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626717184:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827214.422800:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827214.422811:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827214/real 1787827214]  req@ffff8cd0c3da9680 x1874671626717184/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827239 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827214.422823:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626717184/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827239 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827214.422831:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827214.422835:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827214.422838:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626717184:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827214.614187:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827214.998238:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827215.638194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827216.022229:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827216.662188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827217.046213:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827217.686202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827218.070192:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827218.710209:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827219.094252:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827219.542191:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827219.542199:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827219.542202:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827219.542210:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1105
00000100:00080000:3.0:1787827219.542219:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827219.542224:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827219.542256:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626717312/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827219.542277:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827219.542280:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827219.542281:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827219.542283:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1105
00000100:00080000:3.0:1787827219.542286:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827219.542288:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827219.542296:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827219.542306:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626717440/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827219.542315:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1787827219.542320:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626717312:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827219.542320:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827219.542335:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827219.542339:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1105
00000100:00080000:3.0:1787827219.542343:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827219.542346:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827219.542350:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827219.542360:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626717568/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827219.542367:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827219.542371:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827219.542372:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1787827219.542377:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787827219.542379:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1105
00000100:00080000:3.0:1787827219.542385:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827219.542390:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827219.542396:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827219.542408:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626717696/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787827219.542417:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827219/real 1787827219]  req@ffff8cd0c8eac000 x1874671626717312/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827244 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827219.542433:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626717312/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827244 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827219.542448:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827219.542452:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827219.542455:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626717312:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827219.542473:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626717440:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827219.542491:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827219.542499:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827219/real 1787827219]  req@ffff8cd0c8eacf00 x1874671626717440/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827244 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827219.542507:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626717440/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827244 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827219.542512:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827219.542515:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827219.542517:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626717440:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787827219.542520:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626717568:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787827219.542531:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827219.542537:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827219/real 1787827219]  req@ffff8cd0c8eac3c0 x1874671626717568/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827244 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827219.542544:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626717568/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827244 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827219.542549:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827219.542551:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827219.542553:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626717568:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787827219.542562:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626717696:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827219.542575:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827219.542582:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827219/real 1787827219]  req@ffff8cd0c8eae1c0 x1874671626717696/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827244 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827219.542589:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626717696/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827244 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827219.542594:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827219.542596:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827219.542597:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626717696:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827219.734188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827220.118233:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827220.758175:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827221.142272:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827221.782203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827222.166218:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827222.806192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827223.190280:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827223.830188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827224.214245:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827224.662187:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827224.662198:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827224.662215:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827224.662223:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1110
00000100:00080000:3.0:1787827224.662231:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827224.662236:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827224.662264:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626717824/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827224.662284:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827224.662287:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827224.662288:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827224.662291:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1110
00000100:00080000:3.0:1787827224.662293:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827224.662296:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827224.662305:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827224.662314:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626717824:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827224.662315:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626717952/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827224.662327:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827224.662330:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827224.662331:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827224.662333:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1110
00000100:00080000:3.0:1787827224.662336:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827224.662338:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827224.662341:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827224.662349:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626718080/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827224.662366:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827224.662369:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827224.662371:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1787827224.662374:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787827224.662387:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1110
00000100:00080000:3.0:1787827224.662393:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827224.662398:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827224.662404:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827224.662414:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827224/real 1787827224]  req@ffff8cd0c8ead2c0 x1874671626717824/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827249 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827224.662420:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626718208/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787827224.662444:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626717824/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827249 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827224.662453:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827224.662457:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827224.662460:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626717824:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827224.662478:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626717952:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827224.662509:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827224.662520:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827224/real 1787827224]  req@ffff8cd0c8eae940 x1874671626717952/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827249 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827224.662529:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626717952/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827249 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827224.662536:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827224.662538:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827224.662541:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626717952:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787827224.662545:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626718080:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787827224.662558:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827224.662568:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827224/real 1787827224]  req@ffff8cd0c8eaf840 x1874671626718080/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827249 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827224.662580:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626718080/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827249 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827224.662588:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827224.662592:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827224.662595:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626718080:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787827224.662602:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626718208:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827224.662623:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827224.662655:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827224/real 1787827224]  req@ffff8cd0c8eac780 x1874671626718208/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827249 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827224.662683:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626718208/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827249 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827224.662689:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827224.662691:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827224.662693:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626718208:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827224.854133:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827225.238258:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827225.878180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827226.262216:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827226.902184:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827227.286251:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827227.926216:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827228.310236:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827228.950172:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827229.334243:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827229.782191:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827229.782202:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827229.782207:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827229.782218:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1115
00000100:00080000:3.0:1787827229.782229:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827229.782238:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827229.782276:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626718336/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827229.782325:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827229.782329:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827229.782332:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827229.782336:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1115
00000100:00080000:3.0:1787827229.782342:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827229.782347:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827229.782357:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827229.782365:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626718336:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827229.782376:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626718464/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827229.782389:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827229.782393:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827229.782396:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827229.782400:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1115
00000100:00080000:3.0:1787827229.782405:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827229.782410:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827229.782416:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827229.782430:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626718592/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787827229.782434:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787827229.782447:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827229.782451:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827229.782454:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827229.782459:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1115
00000100:00080000:3.0:1787827229.782471:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827229.782477:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787827229.782484:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827229/real 1787827229]  req@ffff8cd0c8eada40 x1874671626718336/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827254 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1787827229.782485:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827229.782504:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626718720/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787827229.782512:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626718336/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827254 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827229.782528:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827229.782534:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827229.782540:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626718336:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827229.782567:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626718464:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827229.782592:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827229.782607:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827229/real 1787827229]  req@ffff8cd0c8eaf0c0 x1874671626718464/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827254 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827229.782622:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626718464/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827254 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827229.782639:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827229.782644:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827229.782648:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626718464:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787827229.782655:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626718592:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787827229.782680:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827229.782694:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827229/real 1787827229]  req@ffff8cd0c8eac780 x1874671626718592/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827254 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827229.782707:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626718592/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827254 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827229.782716:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827229.782720:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827229.782724:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626718592:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787827229.782731:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626718720:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827229.782752:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827229.782765:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827229/real 1787827229]  req@ffff8cd0c8eaf840 x1874671626718720/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827254 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827229.782779:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626718720/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827254 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827229.782788:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827229.782792:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827229.782803:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626718720:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827229.974215:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827230.358196:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827230.998162:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827231.382252:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827232.022218:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827232.406207:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827233.046157:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827233.430241:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827234.070320:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827234.454216:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827234.902198:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827234.902207:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827234.902211:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827234.902233:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1120
00000100:00080000:2.0:1787827234.902245:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827234.902252:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827234.902290:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626718848/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827234.902317:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827234.902319:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827234.902321:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827234.902323:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1120
00000100:00080000:2.0:1787827234.902338:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827234.902342:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827234.902350:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827234.902361:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626718976/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787827234.902371:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626718848:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787827234.902371:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827234.902376:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827234.902380:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827234.902385:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1120
00000100:00080000:2.0:1787827234.902387:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827234.902389:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827234.902392:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827234.902412:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626719104/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827234.902418:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827234.902421:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827234.902422:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827234.902424:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1120
00000100:00080000:2.0:1787827234.902426:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827234.902429:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827234.902434:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827234.902443:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626719232/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787827234.902458:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827234.902517:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827234/real 1787827234]  req@ffff8cd0c3da83c0 x1874671626718848/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827259 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827234.902537:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626718848/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827259 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827234.902549:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827234.902554:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827234.902558:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626718848:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827234.902581:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626718976:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827234.902609:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827234.902624:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827234/real 1787827234]  req@ffff8cd0c3daa940 x1874671626718976/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827259 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827234.902639:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626718976/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827259 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827234.902649:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827234.902653:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827234.902658:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626718976:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787827234.902675:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626719104:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787827234.902701:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827234.902715:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827234/real 1787827234]  req@ffff8cd0c3dab0c0 x1874671626719104/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827259 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827234.902738:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626719104/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827259 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827234.902747:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827234.902751:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827234.902755:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626719104:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787827234.902761:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626719232:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827234.902782:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827234.902795:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827234/real 1787827234]  req@ffff8cd0c3da9a40 x1874671626719232/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827259 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827234.902832:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626719232/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827259 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827234.902841:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827234.902845:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827234.902848:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626719232:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827235.094508:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827235.478316:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827236.118176:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827236.502247:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827237.142164:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827237.526237:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827238.166136:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827238.550184:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827239.190173:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827239.574207:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827240.022227:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827240.022237:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827240.022241:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827240.022252:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1125
00000100:00080000:3.0:1787827240.022263:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827240.022271:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827240.022319:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626719360/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827240.022367:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827240.022371:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827240.022373:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827240.022387:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1125
00000100:00080000:3.0:1787827240.022393:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827240.022398:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827240.022409:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827240.022420:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626719360:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827240.022427:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626719488/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827240.022448:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827240.022452:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827240.022454:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827240.022459:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1125
00000100:00080000:3.0:1787827240.022463:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827240.022468:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827240.022475:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827240.022497:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626719616/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787827240.022504:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787827240.022516:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827240.022521:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827240.022523:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827240.022529:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1125
00000100:00080000:3.0:1787827240.022541:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827240.022547:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827240.022554:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827240.022555:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827240/real 1787827240]  req@ffff8cd0c8eae940 x1874671626719360/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827265 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827240.022572:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626719744/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787827240.022592:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626719360/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827265 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827240.022610:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827240.022615:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827240.022620:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626719360:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827240.022645:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626719488:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827240.022671:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827240.022686:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827240/real 1787827240]  req@ffff8cd0c8ead2c0 x1874671626719488/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827265 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827240.022700:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626719488/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827265 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827240.022709:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827240.022712:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827240.022716:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626719488:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787827240.022722:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626719616:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787827240.022744:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827240.022756:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827240/real 1787827240]  req@ffff8cd0c8eae1c0 x1874671626719616/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827265 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827240.022802:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626719616/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827265 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827240.022811:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827240.022814:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827240.022818:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626719616:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787827240.022824:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626719744:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787827240.022848:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827240.022860:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827240/real 1787827240]  req@ffff8cd0c8eac3c0 x1874671626719744/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827265 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787827240.022872:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626719744/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827265 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827240.022880:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787827240.022883:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827240.022887:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626719744:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827240.214158:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827240.598213:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827241.238252:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827241.622218:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827242.262176:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827242.646187:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827243.286192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827243.670221:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827244.310186:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827244.694185:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787827245.142240:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827245.142249:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787827245.142252:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827245.142261:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1130
00000100:00080000:0.0:1787827245.142270:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827245.142293:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787827245.142323:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626719872/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827245.142346:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827245.142349:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827245.142350:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827245.142353:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1130
00000100:00080000:0.0:1787827245.142356:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827245.142360:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827245.142368:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827245.142381:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626720000/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827245.142382:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626719872:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787827245.142397:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827245.142400:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787827245.142401:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827245.142404:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1130
00000100:00080000:0.0:1787827245.142407:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827245.142409:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827245.142430:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827245.142443:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626720128/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827245.142453:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827245.142456:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827245.142464:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:3.0:1787827245.142468:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787827245.142473:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1130
00000100:00080000:0.0:1787827245.142498:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827245.142502:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827245.142507:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827245.142518:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626720256/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827245.142532:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827245/real 1787827245]  req@ffff8cd0c8e9cf00 x1874671626719872/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827270 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827245.142565:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626719872/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827270 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827245.142574:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827245.142578:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827245.142581:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626719872:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827245.142600:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626720000:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827245.142635:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827245.142644:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827245/real 1787827245]  req@ffff8cd0c8e9d680 x1874671626720000/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827270 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827245.142669:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626720000/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827270 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827245.142676:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827245.142678:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827245.142680:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626720000:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827245.142701:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626720128:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827245.142718:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827245.142725:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827245/real 1787827245]  req@ffff8cd0c8e9ed00 x1874671626720128/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827270 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827245.142733:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626720128/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827270 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827245.142752:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827245.142754:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827245.142757:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626720128:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827245.142761:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626720256:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827245.142774:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827245.142780:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827245/real 1787827245]  req@ffff8cd0c8e9f840 x1874671626720256/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827270 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827245.142788:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626720256/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827270 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827245.142818:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827245.142820:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827245.142823:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626720256:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827245.334184:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827245.718207:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827246.358627:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827246.742213:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827247.382233:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827247.766198:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827248.406134:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827248.790320:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827249.430230:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827249.814221:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787827250.262192:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827250.262202:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787827250.262207:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827250.262217:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1136
00000100:00080000:0.0:1787827250.262229:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827250.262237:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787827250.262271:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626720384/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827250.262303:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827250.262307:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827250.262309:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827250.262313:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1136
00000100:00080000:0.0:1787827250.262317:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827250.262321:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827250.262330:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827250.262345:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626720384:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827250.262358:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626720512/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827250.262372:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827250.262376:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787827250.262378:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827250.262382:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1136
00000100:00080000:0.0:1787827250.262386:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827250.262391:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827250.262396:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827250.262420:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626720640/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827250.262430:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827250.262434:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827250.262437:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827250.262444:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1136
00000100:00100000:2.0:1787827250.262445:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626720512:192.168.203.145@tcp:38:kworker.0
00000100:00080000:0.0:1787827250.262448:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827250.262456:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827250.262461:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827250.262473:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626720768/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827250.262492:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626720640:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827250.262514:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626720768:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827250.454154:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827250.838296:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827251.478279:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827251.862226:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827252.502132:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827252.886218:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:1.0:1787827253.334358:0:2346:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:1.0:1787827253.334395:0:2346:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1787827253.334412:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827253.334451:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827253.334461:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827253.334474:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:2.0:1787827253.334506:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827250/real 1787827253]  req@ffff8cd0c8e9c3c0 x1874671626720768/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827275 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827253.334538:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626720768/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827275 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827253.334549:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827253.334554:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827253.334558:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626720768:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827253.334566:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827250/real 1787827253]  req@ffff8cd0c8e9f480 x1874671626720512/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827275 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827253.334579:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626720512/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827275 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827253.334588:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827253.334591:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827253.334595:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626720512:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827253.334602:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827250/real 1787827253]  req@ffff8cd0c8e9cb40 x1874671626720640/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827275 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827253.334630:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626720640/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827275 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827253.334639:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827253.334643:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827253.334646:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626720640:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827253.334653:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827250/real 1787827253]  req@ffff8cd0c8e9c000 x1874671626720384/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827275 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827253.334664:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626720384/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827275 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827253.334672:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827253.334675:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827253.334697:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626720384:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827253.526200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827253.910296:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827254.550205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827254.934207:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827255.382200:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827255.382213:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827255.382216:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827255.382225:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1141
00000100:00080000:2.0:1787827255.382234:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827255.382239:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827255.382269:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626720896/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827255.382294:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827255.382314:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827255.382316:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827255.382321:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1141
00000100:00080000:2.0:1787827255.382325:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827255.382330:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827255.382337:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626720896:192.168.203.145@tcp:250:kworker.0
00000008:00100000:2.0:1787827255.382346:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827255.382359:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626721024/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827255.382370:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827255.382374:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827255.382375:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827255.382378:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1141
00000100:00080000:2.0:1787827255.382382:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827255.382385:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827255.382390:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787827255.382394:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827255.382398:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626721152/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827255.382407:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827255.382410:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827255.382412:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827255.382415:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1141
00000100:00080000:2.0:1787827255.382418:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827255.382422:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827255.382430:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827255/real 1787827255]  req@ffff8cd0c3da8780 x1874671626720896/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827280 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1787827255.382438:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827255.382447:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626720896/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827280 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827255.382448:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626721280/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827255.382458:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827255.382463:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827255.382474:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626720896:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827255.382497:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626721024:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827255.382522:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827255.382536:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827255/real 1787827255]  req@ffff8cd0c3dab840 x1874671626721024/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827280 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827255.382548:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626721024/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827280 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827255.382558:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827255.382561:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827255.382564:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626721024:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827255.382570:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626721152:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827255.382599:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827255.382610:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827255/real 1787827255]  req@ffff8cd0c3daa1c0 x1874671626721152/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827280 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827255.382622:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626721152/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827280 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827255.382630:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827255.382633:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827255.382636:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626721152:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827255.382642:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626721280:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827255.382661:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827255.382698:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827255/real 1787827255]  req@ffff8cd0c3dabc00 x1874671626721280/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827280 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827255.382710:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626721280/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827280 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827255.382720:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827255.382723:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827255.382727:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626721280:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827255.574181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827255.958223:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827256.598194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827256.982257:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827257.622164:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827258.006207:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827258.646180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827259.030205:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827259.670165:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827260.054317:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827260.502227:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827260.502234:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827260.502238:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827260.502246:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1146
00000100:00080000:2.0:1787827260.502256:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827260.502264:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827260.502304:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626721408/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827260.502336:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827260.502339:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827260.502340:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827260.502343:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1146
00000100:00080000:2.0:1787827260.502346:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827260.502349:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827260.502357:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827260.502367:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626721536/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827260.502374:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827260.502377:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827260.502378:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827260.502380:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1146
00000100:00080000:2.0:1787827260.502397:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827260.502400:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827260.502405:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827260.502412:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626721664/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827260.502419:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827260.502422:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827260.502423:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827260.502426:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1146
00000100:00080000:2.0:1787827260.502429:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827260.502432:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827260.502438:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827260.502438:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626721408:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827260.502451:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626721792/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787827260.502499:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827260.502534:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827260/real 1787827260]  req@ffff8cd0c3da8000 x1874671626721408/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827285 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827260.502548:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626721408/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827285 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827260.502556:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827260.502559:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827260.502562:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626721408:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827260.502576:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626721536:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827260.502594:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827260.502602:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827260/real 1787827260]  req@ffff8cd0c3da9e00 x1874671626721536/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827285 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827260.502611:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626721536/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827285 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827260.502616:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827260.502618:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827260.502620:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626721536:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827260.502624:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626721664:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827260.502651:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827260.502659:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827260/real 1787827260]  req@ffff8cd0c3da92c0 x1874671626721664/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827285 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827260.502667:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626721664/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827285 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827260.502673:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827260.502675:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827260.502677:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626721664:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827260.502696:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626721792:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827260.502706:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827260.502713:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827260/real 1787827260]  req@ffff8cd0c3daa580 x1874671626721792/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827285 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827260.502729:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626721792/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827285 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827260.502735:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827260.502737:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827260.502739:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626721792:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827260.694250:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827261.078224:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827261.718169:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827262.102263:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827262.742196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827263.126191:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827263.766176:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827264.150248:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827264.790197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827265.174240:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827265.622152:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827265.622160:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827265.622164:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827265.622172:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1151
00000100:00080000:2.0:1787827265.622181:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827265.622187:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827265.622217:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daad00 x1874671626721920/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827265.622239:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827265.622255:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827265.622257:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827265.622261:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1151
00000100:00080000:2.0:1787827265.622265:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827265.622268:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827265.622276:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827265.622283:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626721920:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827265.622290:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626722048/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827265.622300:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827265.622302:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827265.622304:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827265.622306:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1151
00000100:00080000:2.0:1787827265.622309:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827265.622311:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827265.622315:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827265.622323:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626722176/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827265.622330:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827265.622332:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827265.622334:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827265.622336:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1151
00000100:00080000:2.0:1787827265.622338:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827265.622341:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:3.0:1787827265.622347:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:2.0:1787827265.622351:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827265.622362:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626722304/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827265.622384:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827265/real 1787827265]  req@ffff8cd0c3daad00 x1874671626721920/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827290 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827265.622405:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daad00 x1874671626721920/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827290 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827265.622414:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827265.622417:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827265.622420:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626721920:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827265.622441:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626722048:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827265.622468:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827265.622477:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827265/real 1787827265]  req@ffff8cd0c3daa580 x1874671626722048/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827290 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827265.622486:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626722048/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827290 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827265.622492:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827265.622494:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827265.622496:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626722048:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827265.622500:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626722176:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827265.622511:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827265.622517:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827265/real 1787827265]  req@ffff8cd0c3da92c0 x1874671626722176/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827290 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827265.622525:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626722176/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827290 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827265.622530:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827265.622532:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827265.622534:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626722176:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827265.622537:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626722304:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827265.622546:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827265.622552:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827265/real 1787827265]  req@ffff8cd0c3da9e00 x1874671626722304/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827290 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827265.622569:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626722304/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827290 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827265.622575:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827265.622577:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827265.622578:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626722304:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827265.814229:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827266.198324:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827266.838189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827267.222181:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827267.862230:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827268.246249:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827268.886184:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827269.270169:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827269.910187:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827270.294257:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827270.742200:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827270.742209:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827270.742213:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827270.742221:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1156
00000100:00080000:3.0:1787827270.742232:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827270.742240:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827270.742280:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626722432/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827270.742312:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827270.742315:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827270.742316:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827270.742333:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1156
00000100:00080000:3.0:1787827270.742337:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827270.742341:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827270.742349:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827270.742358:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626722432:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827270.742365:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626722560/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827270.742380:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827270.742383:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827270.742384:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827270.742389:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1156
00000100:00080000:3.0:1787827270.742392:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827270.742394:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827270.742398:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827270.742409:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626722688/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827270.742416:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827270.742419:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827270.742421:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827270.742424:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1156
00000100:00080000:3.0:1787827270.742434:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827270.742438:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827270.742442:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787827270.742457:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827270.742458:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626722816/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827270.742513:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827270/real 1787827270]  req@ffff8cd0c8eacf00 x1874671626722432/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827295 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827270.742530:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626722432/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827295 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827270.742547:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827270.742551:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827270.742555:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626722432:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827270.742596:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626722560:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827270.742619:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827270.742629:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827270/real 1787827270]  req@ffff8cd0c8eac000 x1874671626722560/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827295 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827270.742638:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626722560/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827295 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827270.742644:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827270.742647:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827270.742650:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626722560:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827270.742654:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626722688:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827270.742683:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827270.742691:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827270/real 1787827270]  req@ffff8cd0c8eac3c0 x1874671626722688/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827295 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827270.742699:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626722688/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827295 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827270.742704:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827270.742706:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827270.742708:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626722688:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827270.742712:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626722816:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827270.742724:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827270.742730:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827270/real 1787827270]  req@ffff8cd0c8eae1c0 x1874671626722816/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827295 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827270.742744:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626722816/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827295 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827270.742750:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827270.742753:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827270.742755:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626722816:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827270.934200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827271.318250:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827271.958214:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827272.342315:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827272.982229:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827273.366246:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827274.006184:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827274.390233:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827275.030205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827275.414206:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827275.862195:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827275.862209:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827275.862215:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827275.862227:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1161
00000100:00080000:2.0:1787827275.862242:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827275.862272:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827275.862327:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626722944/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827275.862358:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827275.862368:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827275.862371:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827275.862375:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1161
00000100:00080000:2.0:1787827275.862380:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827275.862385:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827275.862398:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827275.862404:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626722944:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827275.862414:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626723072/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827275.862429:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827275.862433:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827275.862436:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827275.862440:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1161
00000100:00080000:2.0:1787827275.862444:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827275.862448:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827275.862455:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827275.862466:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626723200/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827275.862475:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827275.862479:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000800:00000100:3.0:1787827275.862481:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787827275.862488:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827275.862495:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1161
00000100:00080000:2.0:1787827275.862500:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827275.862506:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827275.862512:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827275.862536:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626723328/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827275.862545:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827275/real 1787827275]  req@ffff8cd0c3da8000 x1874671626722944/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827275.862566:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626722944/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827275.862576:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827275.862580:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827275.862583:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626722944:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827275.862607:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626723072:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827275.862626:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827275.862636:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827275/real 1787827275]  req@ffff8cd0c3dabc00 x1874671626723072/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827275.862645:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626723072/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827275.862651:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827275.862659:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827275.862662:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626723072:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827275.862667:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626723200:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827275.862682:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827275.862690:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827275/real 1787827275]  req@ffff8cd0c3daa1c0 x1874671626723200/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827275.862698:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626723200/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827275.862703:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827275.862705:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827275.862707:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626723200:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827275.862711:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626723328:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827275.862722:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827275.862728:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827275/real 1787827275]  req@ffff8cd0c3dab840 x1874671626723328/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827275.862736:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626723328/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827275.862741:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827275.862743:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827275.862745:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626723328:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827276.054191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827276.438292:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827277.078198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827277.462211:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827278.102241:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827278.486291:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827279.126175:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827279.510185:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827280.150191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827280.534222:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827280.982220:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827280.982230:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827280.982235:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827280.982246:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1166
00000100:00080000:2.0:1787827280.982258:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827280.982267:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827280.982305:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626723456/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827280.982334:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827280.982338:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827280.982341:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827280.982345:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1166
00000100:00080000:2.0:1787827280.982349:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827280.982355:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827280.982365:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827280.982370:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626723456:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827280.982384:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626723584/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827280.982420:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827280.982425:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827280.982427:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827280.982432:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1166
00000800:00000100:3.0:1787827280.982436:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787827280.982441:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827280.982449:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827280.982456:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827280.982472:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626723712/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827280.982473:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827280/real 1787827280]  req@ffff8cd0c3da8780 x1874671626723456/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827280.982514:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787827280.982516:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626723456/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827280.982521:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827280.982526:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827280.982532:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827280.982535:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1166
00000100:00080000:3.0:1787827280.982539:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827280.982543:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626723456:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787827280.982548:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827280.982555:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827280.982561:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827280.982564:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626723584:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827280.982579:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626723840/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787827280.982589:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827280.982604:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827280/real 1787827280]  req@ffff8cd0c3da9a40 x1874671626723584/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827280.982615:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626723584/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827280.982622:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827280.982625:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827280.982633:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626723584:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827280.982638:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626723712:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827280.982654:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827280.982661:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827280/real 1787827280]  req@ffff8cd0c3dab0c0 x1874671626723712/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827280.982669:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626723712/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827280.982674:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827280.982676:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827280.982678:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626723712:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827280.982687:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626723840:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827280.982698:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827280.982704:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827280/real 1787827280]  req@ffff8cd0c3daa940 x1874671626723840/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827280.982712:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626723840/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827280.982717:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827280.982719:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827280.982721:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626723840:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827281.174155:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827281.558175:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827282.198180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827282.582223:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827283.222265:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827283.606262:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827284.246166:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827284.630275:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827285.270177:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827285.654250:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827286.102161:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827286.102171:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827286.102175:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827286.102182:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1171
00000100:00080000:2.0:1787827286.102205:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827286.102211:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827286.102239:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626723968/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827286.102261:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827286.102263:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827286.102265:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827286.102268:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1171
00000100:00080000:2.0:1787827286.102270:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827286.102273:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827286.102282:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827286.102290:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626724096/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827286.102299:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827286.102302:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827286.102305:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1787827286.102308:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626723968:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787827286.102309:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1171
00000100:00080000:2.0:1787827286.102315:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827286.102319:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827286.102323:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827286.102331:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626724224/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827286.102338:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827286.102345:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827286.102347:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827286.102350:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1171
00000100:00080000:2.0:1787827286.102366:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827286.102370:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827286.102376:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787827286.102382:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827286.102393:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab480 x1874671626724352/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827286.102418:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827286/real 1787827286]  req@ffff8cd0c3da83c0 x1874671626723968/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827311 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827286.102436:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626723968/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827311 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827286.102460:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827286.102466:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827286.102474:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626723968:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827286.102500:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626724096:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827286.102525:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827286.102536:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827286/real 1787827286]  req@ffff8cd0c3da9680 x1874671626724096/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827311 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827286.102544:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626724096/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827311 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827286.102550:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827286.102552:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827286.102554:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626724096:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827286.102567:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626724224:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827286.102583:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827286.102591:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827286/real 1787827286]  req@ffff8cd0c3da8f00 x1874671626724224/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827311 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827286.102599:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626724224/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827311 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827286.102604:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827286.102606:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827286.102608:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626724224:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827286.102612:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626724352:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827286.102627:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827286.102638:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827286/real 1787827286]  req@ffff8cd0c3dab480 x1874671626724352/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827311 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827286.102649:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab480 x1874671626724352/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827311 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827286.102658:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827286.102661:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827286.102664:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626724352:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827286.294155:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827286.678269:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827287.318154:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827287.702197:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827288.342346:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827288.726237:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827289.366162:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827289.750290:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827290.390402:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827290.774226:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827291.222174:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827291.222185:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827291.222190:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827291.222200:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1177
00000100:00080000:2.0:1787827291.222214:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827291.222224:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827291.222275:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8b40 x1874671626724480/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827291.222302:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827291.222305:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827291.222307:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827291.222310:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1177
00000100:00080000:2.0:1787827291.222313:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827291.222316:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827291.222326:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827291.222339:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab480 x1874671626724608/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827291.222370:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827291.222374:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827291.222378:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1787827291.222380:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626724480:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787827291.222384:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1177
00000100:00080000:2.0:1787827291.222396:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827291.222401:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827291.222428:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827291.222442:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626724736/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827291.222449:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827291.222452:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827291.222454:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827291.222457:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1177
00000100:00080000:2.0:1787827291.222461:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827291.222465:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827291.222468:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787827291.222474:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827291.222480:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626724864/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827291.222524:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827291/real 1787827291]  req@ffff8cd0c3da8b40 x1874671626724480/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827316 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827291.222538:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8b40 x1874671626724480/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827316 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827291.222546:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827291.222549:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827291.222552:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626724480:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827291.222579:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626724608:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827291.222605:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827291.222619:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827291/real 1787827291]  req@ffff8cd0c3dab480 x1874671626724608/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827316 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827291.222632:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab480 x1874671626724608/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827316 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827291.222641:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827291.222645:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827291.222649:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626724608:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827291.222655:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626724736:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827291.222691:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827291.222700:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827291/real 1787827291]  req@ffff8cd0c3da8f00 x1874671626724736/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827316 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827291.222708:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626724736/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827316 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827291.222713:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827291.222716:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827291.222718:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626724736:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827291.222722:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626724864:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827291.222733:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827291.222739:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827291/real 1787827291]  req@ffff8cd0c3da9680 x1874671626724864/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827316 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827291.222747:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626724864/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827316 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827291.222751:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827291.222753:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827291.222755:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626724864:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827291.414150:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827291.798278:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827292.438186:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827292.822273:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827293.462215:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827293.846258:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827294.486179:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827294.870225:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827295.510150:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827295.894251:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787827296.342235:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827296.342250:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787827296.342256:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827296.342269:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1182
00000100:00080000:0.0:1787827296.342283:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827296.342291:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787827296.342337:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d2c0 x1874671626724992/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827296.342372:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827296.342375:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827296.342406:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827296.342411:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1182
00000100:00080000:0.0:1787827296.342427:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827296.342433:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827296.342447:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827296.342450:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626724992:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827296.342468:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626725120/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827296.342483:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827296.342487:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787827296.342489:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827296.342493:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1182
00000100:00080000:0.0:1787827296.342496:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827296.342499:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827296.342504:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827296.342515:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626725248/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827296.342523:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827296.342525:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827296.342526:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827296.342529:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1182
00000100:00080000:0.0:1787827296.342531:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827296.342534:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827296.342540:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787827296.342540:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827296.342557:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626725376/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827296.342599:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827296/real 1787827296]  req@ffff8cd0c8e9d2c0 x1874671626724992/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827321 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827296.342620:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d2c0 x1874671626724992/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827321 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827296.342643:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827296.342649:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827296.342654:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626724992:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827296.342684:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626725120:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827296.342713:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827296.342737:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827296/real 1787827296]  req@ffff8cd0c8e9c000 x1874671626725120/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827321 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827296.342752:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626725120/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827321 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827296.342762:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827296.342766:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827296.342770:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626725120:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827296.342777:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626725248:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827296.342811:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827296.342821:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827296/real 1787827296]  req@ffff8cd0c8e9cb40 x1874671626725248/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827321 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827296.342839:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626725248/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827321 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827296.342845:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827296.342848:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827296.342850:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626725248:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827296.342854:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626725376:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827296.342870:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827296.342877:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827296/real 1787827296]  req@ffff8cd0c8e9f480 x1874671626725376/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827321 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827296.342886:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626725376/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827321 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827296.342891:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827296.342893:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827296.342896:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626725376:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827296.534903:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827296.918263:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827297.558186:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827297.942233:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827298.582164:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827298.966234:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827299.606157:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827299.990220:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827300.630180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827301.014327:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827301.462219:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827301.462244:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827301.462247:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827301.462255:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1187
00000100:00080000:3.0:1787827301.462265:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827301.462270:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827301.462300:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626725504/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827301.462324:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827301.462327:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827301.462328:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827301.462331:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1187
00000100:00080000:3.0:1787827301.462333:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827301.462335:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827301.462354:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827301.462365:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626725632/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827301.462375:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827301.462378:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827301.462380:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827301.462383:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1187
00000100:00080000:3.0:1787827301.462386:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827301.462389:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827301.462392:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827301.462398:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626725760/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827301.462412:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827301.462416:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827301.462422:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1787827301.462424:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626725504:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827301.462430:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1187
00000100:00080000:3.0:1787827301.462442:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827301.462449:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827301.462456:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827301.462472:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626725888/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787827301.462522:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827301.462580:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827301/real 1787827301]  req@ffff8cd0c8ead2c0 x1874671626725504/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827326 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827301.462623:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626725504/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827326 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827301.462632:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827301.462636:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827301.462639:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626725504:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827301.462646:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626725632:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827301.462667:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827301.462676:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827301/real 1787827301]  req@ffff8cd0c8eae940 x1874671626725632/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827326 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827301.462685:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626725632/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827326 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827301.462690:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827301.462693:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827301.462695:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626725632:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827301.462721:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626725760:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827301.462733:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827301.462740:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827301/real 1787827301]  req@ffff8cd0c8eaf840 x1874671626725760/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827326 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827301.462749:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626725760/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827326 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827301.462770:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827301.462774:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827301.462778:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626725760:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827301.462785:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626725888:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827301.462809:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827301.462819:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827301/real 1787827301]  req@ffff8cd0c8eac780 x1874671626725888/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827326 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827301.462827:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626725888/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827326 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827301.462832:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827301.462834:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827301.462836:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626725888:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827301.654142:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827302.038243:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827302.678155:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827303.062295:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827303.702143:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827304.086236:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827304.726155:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827305.110283:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827305.750170:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827306.134190:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827306.582153:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827306.582162:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827306.582165:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827306.582173:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1192
00000100:00080000:3.0:1787827306.582183:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827306.582188:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827306.582221:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626726016/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827306.582246:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827306.582261:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827306.582262:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827306.582265:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1192
00000100:00080000:3.0:1787827306.582269:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827306.582273:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827306.582281:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827306.582296:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626726144/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827306.582297:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626726016:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827306.582331:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827306.582334:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827306.582336:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827306.582339:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1192
00000100:00080000:3.0:1787827306.582343:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827306.582346:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827306.582351:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827306.582361:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626726272/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827306.582368:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:2.0:1787827306.582374:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787827306.582375:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827306.582379:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827306.582383:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1192
00000100:00080000:3.0:1787827306.582386:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827306.582390:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827306.582399:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827306.582409:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626726400/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827306.582416:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827306/real 1787827306]  req@ffff8cd0c8eaf0c0 x1874671626726016/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827331 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827306.582440:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626726016/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827331 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827306.582451:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827306.582454:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827306.582457:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626726016:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827306.582485:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626726144:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827306.582505:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827306.582521:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827306/real 1787827306]  req@ffff8cd0c8eada40 x1874671626726144/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827331 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827306.582530:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626726144/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827331 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827306.582536:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827306.582539:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827306.582543:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626726144:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827306.582550:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626726272:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827306.582571:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827306.582583:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827306/real 1787827306]  req@ffff8cd0c8eac780 x1874671626726272/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827331 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827306.582594:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626726272/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827331 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827306.582600:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827306.582603:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827306.582605:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626726272:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827306.582609:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626726400:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827306.582624:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827306.582632:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827306/real 1787827306]  req@ffff8cd0c8eaf840 x1874671626726400/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827331 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827306.582646:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626726400/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827331 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827306.582652:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827306.582656:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827306.582659:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626726400:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827306.774138:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827307.158240:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827307.798181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827308.182198:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827308.822162:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827309.206251:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827309.846222:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827310.230178:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827310.870240:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827311.254248:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827311.702174:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827311.702181:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827311.702184:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827311.702193:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1197
00000100:00080000:3.0:1787827311.702203:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827311.702208:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827311.702240:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626726528/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827311.702262:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827311.702265:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827311.702266:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827311.702269:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1197
00000100:00080000:3.0:1787827311.702283:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827311.702288:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827311.702296:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827311.702316:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626726528:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827311.702323:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626726656/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827311.702333:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827311.702335:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827311.702337:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827311.702339:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1197
00000100:00080000:3.0:1787827311.702342:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827311.702345:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827311.702349:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827311.702356:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626726784/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827311.702363:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827311.702365:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827311.702366:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827311.702369:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1197
00000100:00080000:3.0:1787827311.702371:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1787827311.702376:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787827311.702377:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827311.702382:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827311.702389:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626726912/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827311.702425:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827311/real 1787827311]  req@ffff8cd0c8eae940 x1874671626726528/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827336 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827311.702440:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626726528/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827336 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827311.702449:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827311.702452:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827311.702455:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626726528:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827311.702474:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626726656:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827311.702492:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827311.702501:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827311/real 1787827311]  req@ffff8cd0c8ead2c0 x1874671626726656/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827336 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827311.702510:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626726656/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827336 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827311.702526:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827311.702528:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827311.702531:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626726656:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827311.702535:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626726784:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827311.702550:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827311.702557:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827311/real 1787827311]  req@ffff8cd0c8eae1c0 x1874671626726784/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827336 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827311.702565:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626726784/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827336 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827311.702570:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827311.702572:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827311.702574:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626726784:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827311.702578:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626726912:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827311.702588:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827311.702594:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827311/real 1787827311]  req@ffff8cd0c8eac3c0 x1874671626726912/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827336 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827311.702601:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626726912/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827336 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827311.702606:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827311.702608:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827311.702615:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626726912:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827311.894259:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827312.278248:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827312.918152:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827313.302250:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827313.942182:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827314.326177:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827314.966222:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827315.350273:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827315.990171:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827316.374232:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787827316.822229:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827316.822250:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787827316.822255:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827316.822264:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1202
00000100:00080000:0.0:1787827316.822276:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827316.822282:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787827316.822322:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626727040/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827316.822348:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827316.822352:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827316.822354:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827316.822357:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1202
00000100:00080000:0.0:1787827316.822361:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827316.822365:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827316.822390:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827316.822395:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626727040:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827316.822407:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626727168/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827316.822422:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827316.822426:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787827316.822428:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827316.822433:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1202
00000100:00080000:0.0:1787827316.822438:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827316.822443:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827316.822449:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827316.822461:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626727296/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827316.822472:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827316.822476:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827316.822478:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827316.822482:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1202
00000100:00080000:0.0:1787827316.822486:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827316.822490:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827316.822495:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827316.822507:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626727168:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787827316.822516:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626727424/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827316.822562:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626727296:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827316.822597:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626727424:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827317.014204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827317.398200:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827318.038145:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827318.422216:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827319.062276:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827319.446255:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:2.0:1787827319.894503:0:2343:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:2.0:1787827319.894565:0:2343:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:2.0:1787827319.894582:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787827319.894628:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787827319.894637:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787827319.894645:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:3.0:1787827319.894714:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827316/real 1787827319]  req@ffff8cd0c8e9d680 x1874671626727424/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827319.894764:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626727424/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827319.894780:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827319.894786:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827319.894792:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626727424:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827319.894803:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827316/real 1787827319]  req@ffff8cd0c8e9f840 x1874671626727168/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827319.894818:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626727168/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827319.894828:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827319.894832:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827319.894835:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626727168:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827319.894843:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827316/real 1787827319]  req@ffff8cd0c8e9ed00 x1874671626727296/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827319.894889:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626727296/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827319.894898:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827319.894902:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827319.894905:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626727296:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827319.894915:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827316/real 1787827319]  req@ffff8cd0c8e9c3c0 x1874671626727040/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827319.894929:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626727040/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827319.894936:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827319.894940:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827319.894944:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626727040:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827320.086168:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827320.470259:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827321.110188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827321.494271:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827321.942228:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827321.942236:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827321.942240:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827321.942249:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1207
00000100:00080000:3.0:1787827321.942257:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827321.942263:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827321.942296:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626727552/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827321.942343:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827321.942346:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827321.942350:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827321.942353:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1207
00000100:00080000:3.0:1787827321.942357:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827321.942360:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827321.942368:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827321.942378:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626727680/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827321.942386:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827321.942389:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827321.942390:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827321.942393:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1207
00000100:00080000:3.0:1787827321.942395:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1787827321.942401:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626727552:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827321.942403:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827321.942413:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827321.942435:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626727808/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827321.942446:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827321.942450:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827321.942451:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827321.942456:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1207
00000100:00080000:3.0:1787827321.942468:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1787827321.942475:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787827321.942476:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827321.942485:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827321.942500:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626727936/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827321.942525:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827321/real 1787827321]  req@ffff8cd0c8eac000 x1874671626727552/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827321.942546:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626727552/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827321.942557:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827321.942561:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827321.942565:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626727552:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827321.942572:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626727680:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827321.942610:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827321.942626:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827321/real 1787827321]  req@ffff8cd0c8eacf00 x1874671626727680/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827321.942640:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626727680/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827321.942649:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827321.942653:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827321.942656:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626727680:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827321.942681:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626727808:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827321.942705:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827321.942725:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827321/real 1787827321]  req@ffff8cd0c8eac3c0 x1874671626727808/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827321.942737:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626727808/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827321.942744:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827321.942747:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827321.942750:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626727808:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827321.942756:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626727936:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827321.942775:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827321.942786:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827321/real 1787827321]  req@ffff8cd0c8eae1c0 x1874671626727936/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827321.942812:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626727936/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827321.942820:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827321.942823:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827321.942826:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626727936:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827322.134140:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827322.518239:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827323.158209:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827323.542211:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827324.182149:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827324.566205:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827325.206153:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827325.590246:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827326.230201:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827326.614219:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827327.062270:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827327.062281:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827327.062286:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827327.062297:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1212
00000100:00080000:3.0:1787827327.062310:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827327.062329:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827327.062378:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626728064/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827327.062407:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827327.062410:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827327.062411:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827327.062414:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1212
00000100:00080000:3.0:1787827327.062416:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827327.062419:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827327.062461:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827327.062475:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626728192/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827327.062486:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827327.062489:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827327.062501:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827327.062506:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1212
00000100:00080000:3.0:1787827327.062512:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1787827327.062514:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626728064:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827327.062519:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827327.062526:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827327.062537:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626728320/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827327.062543:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827327.062547:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827327.062549:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827327.062552:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1212
00000100:00080000:3.0:1787827327.062554:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827327.062557:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827327.062560:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827327.062568:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626728448/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787827327.062612:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827327.062687:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827327/real 1787827327]  req@ffff8cd0c8ead2c0 x1874671626728064/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827352 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827327.062703:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626728064/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827352 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827327.062729:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827327.062733:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827327.062737:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626728064:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827327.062764:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626728192:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827327.062790:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827327.062806:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827327/real 1787827327]  req@ffff8cd0c8eae940 x1874671626728192/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827352 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827327.062825:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626728192/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827352 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827327.062864:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827327.062871:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827327.062876:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626728192:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827327.062883:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626728320:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827327.062931:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827327.062950:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827327/real 1787827327]  req@ffff8cd0c8eaf840 x1874671626728320/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827352 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827327.062967:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626728320/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827352 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827327.062975:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827327.062978:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827327.062981:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626728320:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827327.062988:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626728448:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827327.063020:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827327.063032:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827327/real 1787827327]  req@ffff8cd0c8eac780 x1874671626728448/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827352 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827327.063043:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626728448/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827352 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827327.063050:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827327.063053:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827327.063063:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626728448:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827327.254283:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827327.638234:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827328.278289:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827328.662267:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827329.302253:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827329.686211:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827330.326195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827330.710220:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827331.350217:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827331.734254:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827332.182220:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827332.182232:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827332.182236:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827332.182244:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1218
00000100:00080000:1.0:1787827332.182253:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827332.182259:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827332.182293:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626728576/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827332.182314:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827332.182317:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827332.182318:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827332.182321:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1218
00000100:00080000:1.0:1787827332.182323:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827332.182327:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827332.182336:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827332.182368:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626728704/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827332.182382:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626728576:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787827332.182389:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827332.182392:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827332.182394:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827332.182398:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1218
00000100:00080000:1.0:1787827332.182401:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827332.182405:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827332.182410:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827332.182421:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626728832/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827332.182429:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827332.182432:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827332.182433:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827332.182436:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1218
00000100:00080000:1.0:1787827332.182438:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827332.182441:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827332.182444:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827332.182451:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626728960/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787827332.182463:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827332.182514:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827332/real 1787827332]  req@ffff8cd0d48f61c0 x1874671626728576/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827357 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827332.182529:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626728576/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827357 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827332.182538:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827332.182542:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827332.182545:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626728576:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827332.182571:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626728704:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827332.182588:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827332.182597:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827332/real 1787827332]  req@ffff8cd0d48f7c00 x1874671626728704/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827357 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827332.182605:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626728704/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827357 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827332.182610:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827332.182612:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827332.182614:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626728704:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827332.182618:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626728832:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827332.182631:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827332.182650:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827332/real 1787827332]  req@ffff8cd0d48f52c0 x1874671626728832/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827357 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827332.182663:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626728832/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827357 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827332.182671:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827332.182675:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827332.182678:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626728832:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827332.182685:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626728960:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827332.182708:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827332.182718:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827332/real 1787827332]  req@ffff8cd0d48f6d00 x1874671626728960/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827357 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827332.182726:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626728960/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827357 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827332.182732:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827332.182734:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827332.182736:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626728960:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827332.374212:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827332.758325:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827333.398177:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827333.782375:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827334.422219:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827334.806228:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827335.446152:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827335.830290:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827336.470269:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827336.854232:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827337.302225:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827337.302232:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827337.302236:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827337.302258:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1223
00000100:00080000:1.0:1787827337.302269:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827337.302277:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827337.302322:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626729088/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827337.302343:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827337.302350:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827337.302352:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827337.302355:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1223
00000100:00080000:1.0:1787827337.302358:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827337.302361:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827337.302370:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827337.302380:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626729216/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827337.302388:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827337.302391:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827337.302392:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827337.302394:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1223
00000100:00080000:1.0:1787827337.302397:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827337.302401:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827337.302405:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827337.302411:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626729344/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827337.302425:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626729088:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787827337.302431:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827337.302434:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827337.302436:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827337.302442:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1223
00000100:00080000:1.0:1787827337.302453:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827337.302458:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827337.302462:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827337.302472:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626729472/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787827337.302536:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827337.302589:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827337/real 1787827337]  req@ffff8cd0d48f6d00 x1874671626729088/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827362 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827337.302607:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626729088/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827362 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827337.302633:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827337.302639:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827337.302650:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626729088:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827337.302658:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626729216:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827337.302684:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827337.302699:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827337/real 1787827337]  req@ffff8cd0d48f52c0 x1874671626729216/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827362 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827337.302713:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626729216/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827362 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827337.302723:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827337.302727:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827337.302730:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626729216:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827337.302759:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626729344:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827337.302777:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827337.302785:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827337/real 1787827337]  req@ffff8cd0d48f7c00 x1874671626729344/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827362 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827337.302793:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626729344/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827362 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827337.302799:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827337.302801:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827337.302803:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626729344:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827337.302814:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626729472:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827337.302828:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827337.302835:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827337/real 1787827337]  req@ffff8cd0d48f61c0 x1874671626729472/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827362 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827337.302843:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626729472/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827362 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827337.302848:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827337.302850:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827337.302852:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626729472:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827337.494156:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827337.878303:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827338.518222:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827338.902246:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827339.542161:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827339.926305:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827340.566163:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827340.950229:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827341.590173:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827341.976495:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827342.422166:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827342.422175:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827342.422178:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827342.422187:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1228
00000100:00080000:1.0:1787827342.422236:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827342.422243:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827342.422277:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626729600/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827342.422302:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827342.422306:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827342.422308:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827342.422313:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1228
00000100:00080000:1.0:1787827342.422317:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827342.422322:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827342.422346:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827342.422355:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626729600:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787827342.422361:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626729728/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827342.422374:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827342.422377:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827342.422379:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827342.422383:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1228
00000100:00080000:1.0:1787827342.422387:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827342.422391:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827342.422396:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827342.422406:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626729856/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827342.422433:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:2.0:1787827342.422433:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787827342.422437:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827342.422439:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827342.422443:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1228
00000100:00080000:1.0:1787827342.422446:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827342.422450:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827342.422455:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827342.422465:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626729984/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827342.422475:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827342/real 1787827342]  req@ffff8cd0d48f70c0 x1874671626729600/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827367 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827342.422494:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626729600/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827367 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827342.422523:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827342.422528:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827342.422533:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626729600:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827342.422556:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626729728:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827342.422594:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827342.422606:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827342/real 1787827342]  req@ffff8cd0d48f43c0 x1874671626729728/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827367 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827342.422625:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626729728/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827367 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827342.422632:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827342.422635:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827342.422638:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626729728:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827342.422643:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626729856:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827342.422661:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827342.422669:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827342/real 1787827342]  req@ffff8cd0d48f5a40 x1874671626729856/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827367 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827342.422679:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626729856/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827367 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827342.422685:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827342.422687:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827342.422690:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626729856:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827342.422694:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626729984:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827342.422707:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827342.422714:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827342/real 1787827342]  req@ffff8cd0d48f7840 x1874671626729984/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827367 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827342.422723:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626729984/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827367 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827342.422744:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827342.422749:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827342.422754:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626729984:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827342.614242:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827342.998177:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827343.638205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827344.022273:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827344.662206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827345.046198:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827345.686192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827346.070303:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827346.710189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827347.094197:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827347.542155:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827347.542163:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827347.542182:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827347.542190:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1233
00000100:00080000:1.0:1787827347.542200:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827347.542205:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827347.542237:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626730112/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827347.542267:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827347.542270:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827347.542273:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827347.542277:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1233
00000100:00080000:1.0:1787827347.542281:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827347.542284:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827347.542292:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827347.542303:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626730240/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827347.542312:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1787827347.542315:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626730112:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787827347.542316:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827347.542319:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827347.542325:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1233
00000100:00080000:1.0:1787827347.542328:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827347.542329:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827347.542333:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827347.542341:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626730368/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827347.542358:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827347.542360:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827347.542362:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827347.542365:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1233
00000100:00080000:1.0:1787827347.542370:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827347.542372:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827347.542376:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787827347.542380:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787827347.542389:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626730496/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827347.542425:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827347/real 1787827347]  req@ffff8cd0d48f4000 x1874671626730112/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827372 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827347.542440:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626730112/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827372 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827347.542448:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827347.542451:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827347.542454:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626730112:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827347.542472:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626730240:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827347.542487:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827347.542495:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827347/real 1787827347]  req@ffff8cd0d48f6580 x1874671626730240/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827372 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827347.542502:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626730240/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827372 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827347.542508:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827347.542510:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827347.542512:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626730240:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827347.542530:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626730368:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827347.542545:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827347.542553:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827347/real 1787827347]  req@ffff8cd0d48f7480 x1874671626730368/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827372 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827347.542560:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626730368/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827372 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827347.542570:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827347.542572:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827347.542575:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626730368:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827347.542578:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626730496:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827347.542589:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827347.542595:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827347/real 1787827347]  req@ffff8cd0d48f4b40 x1874671626730496/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827372 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827347.542602:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626730496/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827372 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827347.542607:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827347.542609:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827347.542610:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626730496:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827347.734168:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827348.118301:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827348.758216:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827349.142211:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827349.782233:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827350.166365:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827350.806191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827351.190198:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827351.830218:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827352.214240:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827352.662183:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827352.662191:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827352.662195:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827352.662214:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1238
00000100:00080000:2.0:1787827352.662223:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827352.662229:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827352.662260:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626730624/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827352.662283:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827352.662286:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827352.662287:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827352.662290:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1238
00000100:00080000:2.0:1787827352.662292:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827352.662294:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827352.662308:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827352.662323:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626730752/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827352.662336:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787827352.662337:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626730624:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787827352.662344:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827352.662348:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827352.662353:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1238
00000100:00080000:2.0:1787827352.662357:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827352.662361:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827352.662366:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827352.662377:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626730880/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827352.662395:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827352.662398:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827352.662399:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827352.662402:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1238
00000100:00080000:2.0:1787827352.662406:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827352.662410:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:3.0:1787827352.662415:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:2.0:1787827352.662417:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827352.662427:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626731008/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827352.662461:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827352/real 1787827352]  req@ffff8cd0c3da83c0 x1874671626730624/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827377 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827352.662484:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626730624/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827377 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827352.662497:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827352.662501:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827352.662505:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626730624:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827352.662530:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626730752:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827352.662556:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827352.662572:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827352/real 1787827352]  req@ffff8cd0c3daa940 x1874671626730752/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827377 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827352.662587:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626730752/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827377 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827352.662597:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827352.662601:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827352.662605:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626730752:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827352.662612:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626730880:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827352.662645:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827352.662658:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827352/real 1787827352]  req@ffff8cd0c3dab0c0 x1874671626730880/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827377 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827352.662671:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626730880/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827377 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827352.662679:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827352.662683:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827352.662686:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626730880:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827352.662692:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626731008:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827352.662711:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827352.662721:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827352/real 1787827352]  req@ffff8cd0c3da9a40 x1874671626731008/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827377 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827352.662733:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626731008/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827377 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827352.662740:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827352.662743:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827352.662746:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626731008:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827352.854190:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827353.238212:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827353.878200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827354.262269:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827354.902182:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827355.286235:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827355.926210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827356.310242:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827356.950181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827357.334257:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827357.782224:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827357.782234:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827357.782239:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827357.782251:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1243
00000100:00080000:2.0:1787827357.782264:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827357.782272:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827357.782317:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626731136/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827357.782344:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827357.782347:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827357.782349:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827357.782352:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1243
00000100:00080000:2.0:1787827357.782354:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827357.782356:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827357.782386:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827357.782397:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626731264/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827357.782407:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827357.782411:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827357.782414:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1787827357.782414:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626731136:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787827357.782422:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1243
00000100:00080000:2.0:1787827357.782425:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827357.782428:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827357.782432:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827357.782441:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626731392/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827357.782448:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827357.782451:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827357.782452:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827357.782455:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1243
00000100:00080000:2.0:1787827357.782457:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827357.782460:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827357.782463:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827357.782469:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626731520/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787827357.782491:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827357.782544:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827357/real 1787827357]  req@ffff8cd0c3da8780 x1874671626731136/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827382 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827357.782565:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626731136/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827382 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827357.782579:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827357.782584:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827357.782587:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626731136:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827357.782610:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626731264:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827357.782630:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827357.782640:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827357/real 1787827357]  req@ffff8cd0c3dab840 x1874671626731264/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827382 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827357.782649:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626731264/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827382 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827357.782655:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827357.782657:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827357.782659:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626731264:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827357.782663:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626731392:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827357.782675:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827357.782681:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827357/real 1787827357]  req@ffff8cd0c3daa1c0 x1874671626731392/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827382 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827357.782697:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626731392/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827382 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827357.782702:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827357.782705:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827357.782707:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626731392:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827357.782710:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626731520:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827357.782723:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827357.782730:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827357/real 1787827357]  req@ffff8cd0c3dabc00 x1874671626731520/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827382 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827357.782737:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626731520/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827382 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827357.782742:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827357.782744:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827357.782746:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626731520:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827357.974249:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827358.358254:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827358.998174:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827359.382264:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827360.022208:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827360.406307:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827361.046148:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827361.430262:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827362.070206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827362.454261:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787827362.902179:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827362.902187:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787827362.902190:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827362.902198:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1248
00000100:00080000:0.0:1787827362.902213:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827362.902219:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787827362.902249:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626731648/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827362.902268:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827362.902284:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827362.902285:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827362.902289:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1248
00000100:00080000:0.0:1787827362.902292:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827362.902295:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827362.902302:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827362.902311:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626731776/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827362.902319:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827362.902322:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787827362.902324:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827362.902327:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1248
00000100:00080000:0.0:1787827362.902329:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827362.902332:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827362.902335:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827362.902342:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626731904/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827362.902350:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827362.902354:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827362.902359:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1787827362.902361:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626731648:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787827362.902364:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1248
00000100:00080000:0.0:1787827362.902367:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827362.902370:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827362.902384:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827362.902402:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626732032/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787827362.902470:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827362.902516:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827362/real 1787827362]  req@ffff8cd0c8e9cf00 x1874671626731648/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827387 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827362.902531:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626731648/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827387 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827362.902541:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827362.902544:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827362.902549:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626731648:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827362.902557:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626731776:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827362.902582:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827362.902596:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827362/real 1787827362]  req@ffff8cd0c8e9e940 x1874671626731776/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827387 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827362.902610:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626731776/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827387 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827362.902619:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827362.902622:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827362.902624:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626731776:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827362.902654:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626731904:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827362.902672:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827362.902680:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827362/real 1787827362]  req@ffff8cd0c8e9e580 x1874671626731904/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827387 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827362.902700:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626731904/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827387 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827362.902706:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827362.902708:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827362.902710:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626731904:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827362.902714:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626732032:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827362.902728:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827362.902735:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827362/real 1787827362]  req@ffff8cd0c8e9de00 x1874671626732032/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827387 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827362.902742:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626732032/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827387 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827362.902747:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827362.902749:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827362.902751:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626732032:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827363.094154:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827363.478289:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827364.118179:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827364.502213:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827365.142202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827365.526292:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827366.166197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827366.550237:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827367.190182:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827367.574225:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827368.022174:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827368.022202:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827368.022206:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827368.022227:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1253
00000100:00080000:3.0:1787827368.022237:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827368.022243:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827368.022272:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626732160/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827368.022297:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827368.022300:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827368.022301:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827368.022304:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1253
00000100:00080000:3.0:1787827368.022306:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827368.022308:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827368.022317:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827368.022325:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626732288/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827368.022333:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827368.022335:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827368.022337:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827368.022339:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1253
00000100:00080000:3.0:1787827368.022341:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827368.022351:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827368.022354:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827368.022360:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626732160:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827368.022365:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626732416/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827368.022385:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827368.022388:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827368.022389:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827368.022393:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1253
00000100:00080000:3.0:1787827368.022396:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827368.022399:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827368.022402:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827368.022420:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626732544/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787827368.022444:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827368.022476:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827368/real 1787827368]  req@ffff8cd0c8eada40 x1874671626732160/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827393 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827368.022490:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626732160/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827393 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827368.022497:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827368.022500:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827368.022503:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626732160:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827368.022509:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626732288:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827368.022523:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827368.022530:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827368/real 1787827368]  req@ffff8cd0c8eaf0c0 x1874671626732288/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827393 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827368.022538:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626732288/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827393 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827368.022543:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827368.022545:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827368.022547:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626732288:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827368.022566:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626732416:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827368.022578:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827368.022584:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827368/real 1787827368]  req@ffff8cd0c8eac780 x1874671626732416/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827393 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827368.022609:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626732416/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827393 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827368.022614:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827368.022616:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827368.022619:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626732416:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827368.022622:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626732544:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827368.022636:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827368.022643:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827368/real 1787827368]  req@ffff8cd0c8eaf840 x1874671626732544/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827393 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827368.022661:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626732544/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827393 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827368.022666:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827368.022668:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827368.022670:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626732544:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827368.214124:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827368.598327:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827369.238166:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827369.622199:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827370.262175:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827370.646313:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827371.286163:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827371.670229:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827372.310183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827372.694276:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827373.142213:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827373.142224:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827373.142228:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827373.142237:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1258
00000100:00080000:1.0:1787827373.142247:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827373.142254:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827373.142302:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626732672/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827373.142325:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827373.142328:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827373.142330:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827373.142332:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1258
00000100:00080000:1.0:1787827373.142335:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827373.142338:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827373.142358:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827373.142375:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626732800/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827373.142375:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626732672:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787827373.142392:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827373.142395:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827373.142397:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827373.142400:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1258
00000100:00080000:1.0:1787827373.142403:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827373.142406:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827373.142410:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827373.142419:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626732928/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827373.142428:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827373.142431:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827373.142432:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827373.142434:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1258
00000100:00080000:1.0:1787827373.142438:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1787827373.142442:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787827373.142446:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827373.142451:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827373.142460:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626733056/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827373.142486:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827373/real 1787827373]  req@ffff8cd0d48f4b40 x1874671626732672/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827398 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827373.142507:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626732672/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827398 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827373.142535:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827373.142539:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827373.142542:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626732672:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827373.142566:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626732800:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827373.142596:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827373.142613:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827373/real 1787827373]  req@ffff8cd0d48f7480 x1874671626732800/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827398 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827373.142627:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626732800/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827398 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827373.142633:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827373.142636:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827373.142638:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626732800:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827373.142643:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626732928:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827373.142661:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827373.142669:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827373/real 1787827373]  req@ffff8cd0d48f6580 x1874671626732928/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827398 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827373.142677:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626732928/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827398 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827373.142683:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827373.142686:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827373.142689:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626732928:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827373.142704:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626733056:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827373.142742:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827373.142755:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827373/real 1787827373]  req@ffff8cd0d48f4000 x1874671626733056/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827398 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827373.142765:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626733056/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827398 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827373.142771:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827373.142773:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827373.142776:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626733056:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827373.334132:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827373.718213:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827374.358174:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827374.742352:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827375.382150:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827375.766291:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827376.406181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827376.790387:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827377.430160:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827377.814222:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827378.262203:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827378.262229:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827378.262234:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827378.262244:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1264
00000100:00080000:1.0:1787827378.262256:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827378.262264:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827378.262297:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626733184/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827378.262326:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827378.262330:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827378.262332:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827378.262336:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1264
00000100:00080000:1.0:1787827378.262340:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827378.262345:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827378.262356:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827378.262369:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626733312/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827378.262386:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1787827378.262387:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626733184:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787827378.262401:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827378.262405:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827378.262411:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1264
00000100:00080000:1.0:1787827378.262415:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827378.262421:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827378.262427:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827378.262441:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626733440/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827378.262462:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827378.262467:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827378.262469:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827378.262475:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1264
00000800:00000100:2.0:1787827378.262479:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787827378.262487:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827378.262498:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827378.262505:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827378.262521:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626733568/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827378.262533:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827378/real 1787827378]  req@ffff8cd0d48f7840 x1874671626733184/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827403 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827378.262550:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626733184/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827403 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827378.262560:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827378.262564:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827378.262567:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626733184:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827378.262590:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626733312:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827378.262607:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827378.262616:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827378/real 1787827378]  req@ffff8cd0d48f5a40 x1874671626733312/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827403 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827378.262640:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626733312/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827403 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827378.262646:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827378.262648:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827378.262651:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626733312:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827378.262655:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626733440:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827378.262671:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827378.262678:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827378/real 1787827378]  req@ffff8cd0d48f43c0 x1874671626733440/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827403 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827378.262686:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626733440/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827403 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827378.262691:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827378.262692:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827378.262694:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626733440:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827378.262698:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626733568:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827378.262709:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827378.262715:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827378/real 1787827378]  req@ffff8cd0d48f70c0 x1874671626733568/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827403 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827378.262722:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626733568/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827403 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827378.262737:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827378.262739:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827378.262742:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626733568:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827378.454179:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827378.838284:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827379.478220:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827379.862194:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827380.502311:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827380.886307:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827381.526186:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827381.910183:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827382.550183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827382.934324:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827383.382225:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827383.382238:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827383.382245:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827383.382275:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1269
00000100:00080000:3.0:1787827383.382291:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827383.382303:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827383.382367:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626733696/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827383.382406:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827383.382411:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827383.382414:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827383.382421:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1269
00000100:00080000:3.0:1787827383.382428:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827383.382434:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827383.382462:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827383.382474:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626733696:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827383.382481:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626733824/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827383.382495:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827383.382499:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827383.382501:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827383.382505:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1269
00000100:00080000:3.0:1787827383.382508:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827383.382512:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827383.382528:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827383.382542:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626733952/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827383.382553:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827383.382556:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827383.382558:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827383.382562:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1269
00000100:00080000:3.0:1787827383.382567:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827383.382571:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827383.382576:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827383.382585:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626733824:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827383.382632:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626734080/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827383.382662:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626733952:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827383.382745:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626734080:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827383.574197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827383.958227:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827384.598245:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827384.982365:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827385.622180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827386.006213:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:1.0:1787827386.454447:0:2345:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:1.0:1787827386.454501:0:2345:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1787827386.454520:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827386.454583:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827386.454590:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827386.454596:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:2.0:1787827386.454637:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827383/real 1787827386]  req@ffff8cd0c8eac3c0 x1874671626734080/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827408 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827386.454669:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626734080/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827408 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827386.454682:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827386.454687:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827386.454692:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626734080:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827386.454702:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827383/real 1787827386]  req@ffff8cd0c8ead2c0 x1874671626733824/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827408 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827386.454719:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626733824/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827408 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827386.454727:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827386.454745:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827386.454749:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626733824:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827386.454776:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827383/real 1787827386]  req@ffff8cd0c8eae1c0 x1874671626733952/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827408 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827386.454792:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626733952/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827408 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827386.454801:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827386.454805:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827386.454809:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626733952:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827386.454816:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827383/real 1787827386]  req@ffff8cd0c8eae940 x1874671626733696/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827408 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827386.454829:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626733696/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827408 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827386.454839:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827386.454842:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827386.454847:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626733696:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827386.646199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827387.030251:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827387.670216:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827388.054188:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827388.502202:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827388.502210:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827388.502213:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827388.502221:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1274
00000100:00080000:1.0:1787827388.502230:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827388.502247:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827388.502277:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626734208/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827388.502298:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827388.502300:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827388.502302:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827388.502304:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1274
00000100:00080000:1.0:1787827388.502307:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827388.502310:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827388.502317:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827388.502325:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626734336/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827388.502332:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827388.502334:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827388.502335:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827388.502337:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1274
00000100:00080000:1.0:1787827388.502339:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827388.502341:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827388.502343:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827388.502349:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626734464/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827388.502356:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827388.502359:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827388.502364:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827388.502368:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1274
00000100:00080000:1.0:1787827388.502372:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827388.502378:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827388.502381:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626734208:192.168.203.145@tcp:250:kworker.0
00000008:00100000:1.0:1787827388.502385:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827388.502402:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626734592/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787827388.502454:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827388.502494:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827388/real 1787827388]  req@ffff8cd0d48f61c0 x1874671626734208/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827413 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827388.502513:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626734208/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827413 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827388.502533:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827388.502538:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827388.502542:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626734208:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827388.502550:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626734336:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827388.502578:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827388.502591:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827388/real 1787827388]  req@ffff8cd0d48f7c00 x1874671626734336/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827413 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827388.502604:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626734336/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827413 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827388.502612:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827388.502615:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827388.502619:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626734336:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827388.502625:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626734464:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827388.502647:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827388.502659:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827388/real 1787827388]  req@ffff8cd0d48f52c0 x1874671626734464/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827413 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827388.502670:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626734464/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827413 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827388.502678:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827388.502681:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827388.502684:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626734464:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827388.502721:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626734592:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827388.502764:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827388.502777:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827388/real 1787827388]  req@ffff8cd0d48f6d00 x1874671626734592/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827413 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827388.502789:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626734592/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827413 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827388.502803:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827388.502806:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827388.502809:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626734592:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827388.694171:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827389.078304:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827389.718704:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827390.102334:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827390.742228:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827391.126269:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827391.766162:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827392.150197:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827392.790254:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827393.174291:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827393.622197:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827393.622209:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827393.622215:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827393.622226:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1279
00000100:00080000:1.0:1787827393.622240:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827393.622274:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827393.622311:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626734720/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827393.622352:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827393.622356:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827393.622359:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827393.622364:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1279
00000100:00080000:1.0:1787827393.622370:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827393.622375:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827393.622387:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827393.622405:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626734848/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827393.622439:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1787827393.622457:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626734720:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787827393.622466:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827393.622469:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827393.622474:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1279
00000100:00080000:1.0:1787827393.622479:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827393.622483:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827393.622489:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827393.622504:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626734976/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827393.622523:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827393.622527:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827393.622529:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827393.622533:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1279
00000100:00080000:1.0:1787827393.622536:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827393.622539:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:2.0:1787827393.622545:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:1.0:1787827393.622547:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827393.622560:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626735104/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827393.622614:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827393/real 1787827393]  req@ffff8cd0d48f6d00 x1874671626734720/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827418 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827393.622629:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626734720/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827418 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827393.622637:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827393.622640:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827393.622643:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626734720:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827393.622682:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626734848:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827393.622701:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827393.622734:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827393/real 1787827393]  req@ffff8cd0d48f52c0 x1874671626734848/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827418 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827393.622755:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626734848/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827418 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827393.622763:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827393.622784:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827393.622788:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626734848:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827393.622797:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626734976:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827393.622827:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827393.622840:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827393/real 1787827393]  req@ffff8cd0d48f7c00 x1874671626734976/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827418 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827393.622855:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626734976/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827418 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827393.622865:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827393.622873:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827393.622877:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626734976:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827393.622884:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626735104:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827393.622903:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827393.622914:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827393/real 1787827393]  req@ffff8cd0d48f61c0 x1874671626735104/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827418 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827393.622928:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626735104/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827418 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827393.622963:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827393.622968:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827393.622972:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626735104:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827393.814234:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827394.198203:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827394.838206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827395.222236:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827395.862178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827396.246299:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827396.886205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827397.270312:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827397.910159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827398.294236:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827398.742291:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827398.742304:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827398.742309:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827398.742332:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1284
00000100:00080000:2.0:1787827398.742343:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827398.742350:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827398.742387:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626735232/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827398.742416:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827398.742420:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827398.742422:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827398.742425:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1284
00000100:00080000:2.0:1787827398.742427:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827398.742430:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827398.742439:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827398.742457:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626735360/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827398.742466:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827398.742469:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827398.742486:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827398.742490:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1284
00000100:00080000:2.0:1787827398.742495:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827398.742498:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827398.742504:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827398.742517:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626735488/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827398.742517:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626735232:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787827398.742539:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827398.742541:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827398.742543:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827398.742546:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1284
00000100:00080000:2.0:1787827398.742548:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827398.742568:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827398.742573:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827398.742583:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626735616/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787827398.742646:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827398.742708:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827398/real 1787827398]  req@ffff8cd0c3da8000 x1874671626735232/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827423 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827398.742726:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626735232/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827423 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827398.742736:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827398.742739:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827398.742743:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626735232:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827398.742750:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626735360:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827398.742774:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827398.742784:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827398/real 1787827398]  req@ffff8cd0c3da9e00 x1874671626735360/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827423 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827398.742794:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626735360/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827423 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827398.742801:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827398.742804:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827398.742806:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626735360:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827398.742838:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626735488:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827398.742858:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827398.742867:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827398/real 1787827398]  req@ffff8cd0c3da92c0 x1874671626735488/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827423 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827398.742877:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626735488/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827423 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827398.742883:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827398.742885:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827398.742888:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626735488:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827398.742892:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626735616:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827398.742905:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827398.742912:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827398/real 1787827398]  req@ffff8cd0c3daa580 x1874671626735616/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827423 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827398.742921:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626735616/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827423 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827398.742927:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827398.742929:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827398.742931:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626735616:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827398.934224:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827399.318310:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827399.958261:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827400.342205:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827400.982237:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827401.366289:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827402.006164:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827402.390270:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827403.030256:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827403.414385:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827403.862238:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827403.862252:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827403.862257:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827403.862269:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1289
00000100:00080000:2.0:1787827403.862282:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827403.862290:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827403.862336:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daad00 x1874671626735744/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827403.862411:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827403.862418:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827403.862422:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827403.862429:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1289
00000100:00080000:2.0:1787827403.862437:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827403.862444:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827403.862456:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827403.862476:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626735872/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827403.862494:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827403.862501:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827403.862506:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1787827403.862509:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626735744:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787827403.862516:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1289
00000100:00080000:2.0:1787827403.862533:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827403.862556:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827403.862567:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827403.862584:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626736000/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827403.862595:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827403.862599:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827403.862601:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827403.862607:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1289
00000100:00080000:2.0:1787827403.862626:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827403.862657:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827403.862664:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787827403.862675:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827403.862683:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626736128/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827403.862757:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827403/real 1787827403]  req@ffff8cd0c3daad00 x1874671626735744/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827428 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827403.862783:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daad00 x1874671626735744/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827428 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827403.862798:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827403.862803:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827403.862808:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626735744:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827403.862842:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626735872:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827403.862873:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827403.862887:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827403/real 1787827403]  req@ffff8cd0c3daa580 x1874671626735872/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827428 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827403.862901:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626735872/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827428 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827403.862910:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827403.862914:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827403.862917:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626735872:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827403.862924:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626736000:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827403.862944:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827403.862954:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827403/real 1787827403]  req@ffff8cd0c3da92c0 x1874671626736000/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827428 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827403.862999:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626736000/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827428 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827403.863014:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827403.863019:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827403.863026:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626736000:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827403.863037:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626736128:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827403.863077:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827403.863143:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827403/real 1787827403]  req@ffff8cd0c3da9e00 x1874671626736128/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827428 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827403.863157:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626736128/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827428 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827403.863167:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827403.863171:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827403.863174:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626736128:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827404.054237:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827404.438278:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827405.078149:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827405.462265:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827406.102307:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827406.486215:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827407.126289:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827407.510206:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827408.150170:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827408.534224:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787827408.982233:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827408.982242:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787827408.982246:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827408.982256:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1294
00000100:00080000:0.0:1787827408.982291:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827408.982301:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787827408.982346:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626736256/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827408.982383:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827408.982395:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827408.982397:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827408.982401:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1294
00000100:00080000:0.0:1787827408.982405:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827408.982410:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827408.982420:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827408.982436:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626736384/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827408.982451:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626736256:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787827408.982452:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827408.982463:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787827408.982466:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827408.982470:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1294
00000100:00080000:0.0:1787827408.982473:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827408.982476:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827408.982481:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827408.982492:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626736512/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827408.982501:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827408.982511:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827408.982513:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827408.982525:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1294
00000100:00080000:0.0:1787827408.982531:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1787827408.982533:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787827408.982542:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827408.982549:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827408.982564:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626736640/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827408.982587:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827408/real 1787827408]  req@ffff8cd0c8e9fc00 x1874671626736256/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827433 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827408.982636:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626736256/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827433 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827408.982663:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827408.982670:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827408.982676:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626736256:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827408.982704:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626736384:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827408.982731:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827408.982745:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827408/real 1787827408]  req@ffff8cd0c8e9de00 x1874671626736384/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827433 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827408.982756:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626736384/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827433 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827408.982764:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827408.982767:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827408.982770:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626736384:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827408.982775:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626736512:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827408.982790:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827408.982799:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827408/real 1787827408]  req@ffff8cd0c8e9e580 x1874671626736512/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827433 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827408.982809:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626736512/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827433 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827408.982815:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827408.982818:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827408.982820:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626736512:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827408.982833:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626736640:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827408.982851:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827408.982861:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827408/real 1787827408]  req@ffff8cd0c8e9e940 x1874671626736640/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827433 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827408.982871:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626736640/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827433 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827408.982878:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827408.982880:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827408.982883:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626736640:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827409.174219:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827409.558272:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827410.198272:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827410.582282:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827411.222223:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827411.606323:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827412.246205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827412.630265:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827413.270228:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827413.654338:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827414.102308:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827414.102325:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827414.102332:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827414.102347:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1299
00000100:00080000:1.0:1787827414.102366:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827414.102377:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827414.102465:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626736768/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827414.102545:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827414.102550:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827414.102553:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827414.102584:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1299
00000100:00080000:1.0:1787827414.102595:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827414.102605:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827414.102637:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626736768:192.168.203.145@tcp:250:kworker.0
00000008:00100000:1.0:1787827414.102641:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827414.102669:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626736896/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787827414.102718:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787827414.102736:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827414.102742:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827414.102745:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827414.102754:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1299
00000100:00080000:1.0:1787827414.102764:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827414.102777:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827414.102778:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827414/real 1787827414]  req@ffff8cd0d48f70c0 x1874671626736768/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827439 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1787827414.102800:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827414.102825:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626736768/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827439 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827414.102848:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:1.0:1787827414.102852:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626737024/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827414.102859:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827414.102876:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626736768:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787827414.102932:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787827414.102941:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626736896:192.168.203.145@tcp:38:kworker.0
00000100:00080000:1.0:1787827414.102944:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827414.102948:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827414.102956:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1299
00000100:00080000:1.0:1787827414.102963:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827414.102969:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827414.102978:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827414.103000:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626737152/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787827414.103052:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827414.103080:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827414/real 1787827414]  req@ffff8cd0d48f43c0 x1874671626736896/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827439 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827414.103152:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626736896/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827439 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827414.103171:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827414.103175:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827414.103180:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626736896:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827414.103189:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626737024:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827414.103216:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827414.103230:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827414/real 1787827414]  req@ffff8cd0d48f5a40 x1874671626737024/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827439 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827414.103245:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626737024/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827439 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827414.103255:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827414.103258:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827414.103262:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626737024:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827414.103290:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626737152:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827414.103329:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827414.103356:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827414/real 1787827414]  req@ffff8cd0d48f7840 x1874671626737152/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827439 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827414.103416:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626737152/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827439 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827414.103427:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827414.103431:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827414.103435:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626737152:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827414.294266:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827414.678246:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827415.318259:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827415.702263:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827416.342255:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827416.726233:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827417.366276:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827417.750313:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827418.390187:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827418.774246:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827419.222216:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827419.222230:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827419.222237:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827419.222250:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1305
00000100:00080000:3.0:1787827419.222266:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827419.222277:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827419.222323:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626737280/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827419.222376:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827419.222391:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827419.222394:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827419.222400:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1305
00000100:00080000:3.0:1787827419.222408:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827419.222416:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827419.222431:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827419.222433:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626737280:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827419.222487:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626737408/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827419.222502:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827419.222506:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827419.222509:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1787827419.222516:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787827419.222518:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1305
00000100:00080000:3.0:1787827419.222546:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827419.222553:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827419.222588:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827419.222607:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626737536/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827419.222619:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827419.222625:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:2.0:1787827419.222627:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827419/real 1787827419]  req@ffff8cd0c8eacf00 x1874671626737280/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827444 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827419.222628:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827419.222640:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1305
00000100:00080000:3.0:1787827419.222647:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827419.222656:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827419.222663:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827419.222677:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626737280/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827444 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827419.222679:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626737664/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827419.222733:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827419.222744:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827419.222753:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626737280:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827419.222788:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626737408:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827419.222832:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827419.222856:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827419/real 1787827419]  req@ffff8cd0c8eac000 x1874671626737408/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827444 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827419.222879:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626737408/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827444 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827419.222889:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827419.222893:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827419.222924:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626737408:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827419.222931:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626737536:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827419.222958:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827419.222970:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827419/real 1787827419]  req@ffff8cd0c8eae940 x1874671626737536/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827444 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827419.222982:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626737536/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827444 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827419.222990:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827419.222993:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827419.223016:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626737536:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827419.223024:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626737664:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827419.223047:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827419.223060:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827419/real 1787827419]  req@ffff8cd0c8eae1c0 x1874671626737664/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827444 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827419.223074:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626737664/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827444 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827419.223084:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827419.223088:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827419.223092:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626737664:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827419.414178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827419.798334:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827420.438210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827420.822290:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827421.462157:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827421.846354:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827422.486253:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827422.870184:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827423.510199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827423.894266:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787827424.342187:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827424.342196:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787827424.342200:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827424.342209:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1310
00000100:00080000:0.0:1787827424.342219:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827424.342224:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787827424.342277:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626737792/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827424.342299:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827424.342302:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827424.342316:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827424.342319:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1310
00000100:00080000:0.0:1787827424.342323:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827424.342326:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827424.342334:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827424.342344:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626737920/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827424.342353:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827424.342356:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787827424.342358:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827424.342362:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1310
00000100:00080000:0.0:1787827424.342365:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827424.342367:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827424.342370:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827424.342378:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626738048/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827424.342384:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827424.342387:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827424.342388:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827424.342391:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1310
00000100:00080000:0.0:1787827424.342393:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827424.342397:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827424.342400:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827424.342421:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626738176/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827424.342433:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626737792:192.168.203.145@tcp:250:kworker.0
00000800:00000100:2.0:1787827424.342596:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827424.342703:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827424/real 1787827424]  req@ffff8cd0c8e9cf00 x1874671626737792/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827449 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827424.342753:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626737792/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827449 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827424.342769:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827424.342774:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827424.342780:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626737792:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827424.342790:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626737920:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827424.342821:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827424.342836:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827424/real 1787827424]  req@ffff8cd0c8e9c3c0 x1874671626737920/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827449 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827424.342850:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626737920/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827449 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827424.342876:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827424.342884:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827424.342889:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626737920:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827424.342896:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626738048:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827424.342919:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827424.342931:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827424/real 1787827424]  req@ffff8cd0c8e9ed00 x1874671626738048/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827449 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827424.342945:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626738048/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827449 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827424.342972:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827424.342977:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827424.342981:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626738048:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827424.343022:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626738176:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827424.343048:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827424.343061:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827424/real 1787827424]  req@ffff8cd0c8e9f840 x1874671626738176/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827449 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827424.343075:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626738176/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827449 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827424.343127:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827424.343132:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827424.343135:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626738176:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827424.534196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827424.918177:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827425.558206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827425.942322:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827426.582207:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827426.966267:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827427.606212:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827427.990243:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827428.630167:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827429.014203:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787827429.462204:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827429.462214:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787827429.462218:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827429.462229:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1315
00000100:00080000:0.0:1787827429.462240:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827429.462248:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787827429.462288:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626738304/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827429.462318:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827429.462323:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827429.462324:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827429.462329:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1315
00000100:00080000:0.0:1787827429.462333:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827429.462337:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827429.462346:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827429.462360:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626738432/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827429.462374:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1787827429.462374:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626738304:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787827429.462385:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787827429.462401:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827429.462407:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1315
00000100:00080000:0.0:1787827429.462412:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827429.462417:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827429.462424:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787827429.462437:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827429.462442:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626738560/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827429.462474:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827429.462478:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827429.462480:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827429.462485:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1315
00000100:00100000:2.0:1787827429.462488:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827429/real 1787827429]  req@ffff8cd0c8e9d680 x1874671626738304/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827454 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787827429.462491:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827429.462502:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827429.462510:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827429.462511:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626738304/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827454 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827429.462526:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827429.462531:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787827429.462532:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626738688/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827429.462535:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626738304:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827429.462567:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626738432:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827429.462587:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827429.462596:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827429/real 1787827429]  req@ffff8cd0c8e9f480 x1874671626738432/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827454 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827429.462605:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626738432/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827454 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827429.462610:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827429.462612:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827429.462615:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626738432:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827429.462619:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626738560:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827429.462630:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827429.462636:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827429/real 1787827429]  req@ffff8cd0c8e9cb40 x1874671626738560/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827454 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827429.462643:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626738560/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827454 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827429.462648:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827429.462650:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827429.462652:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626738560:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827429.462655:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626738688:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827429.462664:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827429.462670:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827429/real 1787827429]  req@ffff8cd0c8e9c000 x1874671626738688/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827454 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827429.462683:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626738688/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827454 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827429.462689:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827429.462691:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827429.462692:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626738688:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827429.654230:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827430.038240:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827430.678181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827431.062220:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827431.702212:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827432.086327:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827432.726213:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827433.110292:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827433.750220:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827434.134248:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827434.582180:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827434.582188:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827434.582191:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827434.582200:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1320
00000100:00080000:3.0:1787827434.582210:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827434.582215:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827434.582253:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626738816/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827434.582281:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827434.582284:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827434.582285:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827434.582288:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1320
00000100:00080000:3.0:1787827434.582291:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827434.582294:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827434.582302:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827434.582312:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626738944/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827434.582320:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827434.582322:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827434.582323:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827434.582326:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1320
00000100:00080000:3.0:1787827434.582345:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827434.582350:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827434.582355:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626738816:192.168.203.145@tcp:250:kworker.0
00000008:00100000:3.0:1787827434.582358:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827434.582376:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626739072/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827434.582389:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827434.582392:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827434.582393:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827434.582397:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1320
00000100:00080000:3.0:1787827434.582401:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827434.582405:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827434.582409:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787827434.582415:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827434.582432:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626739200/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827434.582627:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827434/real 1787827434]  req@ffff8cd0c8ead2c0 x1874671626738816/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827459 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827434.582659:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626738816/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827459 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827434.582667:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827434.582669:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827434.582672:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626738816:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827434.582678:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626738944:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827434.582696:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827434.582704:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827434/real 1787827434]  req@ffff8cd0c8eac3c0 x1874671626738944/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827459 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827434.582712:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626738944/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827459 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827434.582729:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827434.582732:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827434.582734:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626738944:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827434.582754:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626739072:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827434.582786:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827434.582795:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827434/real 1787827434]  req@ffff8cd0c8eaf840 x1874671626739072/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827459 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827434.582803:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626739072/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827459 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827434.582809:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827434.582812:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827434.582816:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626739072:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827434.582822:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626739200:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827434.582859:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827434.582870:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827434/real 1787827434]  req@ffff8cd0c8eac780 x1874671626739200/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827459 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827434.582883:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626739200/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827459 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827434.582892:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827434.582895:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827434.582904:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626739200:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827434.774156:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827435.158244:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827435.798226:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827436.182232:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827436.822194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827437.206259:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827437.846150:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827438.230287:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827438.870167:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827439.254275:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827439.702204:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827439.702213:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827439.702217:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827439.702224:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1325
00000100:00080000:3.0:1787827439.702247:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827439.702253:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827439.702307:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626739328/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827439.702356:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827439.702359:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827439.702360:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827439.702363:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1325
00000100:00080000:3.0:1787827439.702366:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827439.702368:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827439.702376:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827439.702402:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626739456/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827439.702412:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1787827439.702413:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626739328:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827439.702415:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827439.702419:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827439.702435:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1325
00000100:00080000:3.0:1787827439.702437:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827439.702440:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827439.702444:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827439.702453:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626739584/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827439.702460:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827439.702463:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827439.702464:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827439.702466:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1325
00000100:00080000:3.0:1787827439.702481:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827439.702485:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827439.702489:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787827439.702497:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827439.702500:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626739712/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827439.702550:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827439/real 1787827439]  req@ffff8cd0c8eaf0c0 x1874671626739328/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827464 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827439.702564:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626739328/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827464 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827439.702572:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827439.702575:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827439.702578:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626739328:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827439.702597:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626739456:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827439.702614:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827439.702623:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827439/real 1787827439]  req@ffff8cd0c8eada40 x1874671626739456/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827464 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827439.702631:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626739456/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827464 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827439.702636:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827439.702638:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827439.702640:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626739456:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827439.702644:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626739584:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827439.702655:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827439.702670:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827439/real 1787827439]  req@ffff8cd0c8eac780 x1874671626739584/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827464 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827439.702678:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626739584/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827464 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827439.702684:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827439.702686:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827439.702688:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626739584:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827439.702691:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626739712:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827439.702703:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827439.702709:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827439/real 1787827439]  req@ffff8cd0c8eaf840 x1874671626739712/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827464 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827439.702716:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626739712/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827464 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827439.702721:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827439.702723:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827439.702725:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626739712:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827439.894208:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827440.278222:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827440.918170:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827441.302239:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827441.942166:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827442.326208:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827442.966276:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827443.350226:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827443.990233:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827444.374252:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787827444.822273:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827444.822292:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787827444.822325:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827444.822343:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1330
00000100:00080000:0.0:1787827444.822398:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827444.822408:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787827444.822457:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d2c0 x1874671626739840/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827444.822496:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827444.822501:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827444.822504:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827444.822510:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1330
00000100:00080000:0.0:1787827444.822516:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827444.822522:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827444.822546:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827444.822568:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626739840:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827444.822573:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626739968/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827444.822594:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827444.822599:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787827444.822602:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827444.822607:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1330
00000100:00080000:0.0:1787827444.822614:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827444.822620:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827444.822628:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827444.822645:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626740096/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787827444.822654:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787827444.822683:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827444.822689:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827444.822692:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827444.822699:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1330
00000100:00080000:0.0:1787827444.822706:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827444.822714:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827444.822720:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827444/real 1787827444]  req@ffff8cd0c8e9d2c0 x1874671626739840/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827469 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:0.0:1787827444.822723:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827444.822754:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626740224/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827444.822754:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d2c0 x1874671626739840/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827469 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827444.822786:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827444.822790:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827444.822795:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626739840:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827444.822823:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626739968:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827444.822848:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827444.822868:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827444/real 1787827444]  req@ffff8cd0c8e9c000 x1874671626739968/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827469 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827444.822879:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626739968/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827469 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827444.822886:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827444.822890:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827444.822892:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626739968:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827444.822897:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626740096:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827444.822914:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827444.822922:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827444/real 1787827444]  req@ffff8cd0c8e9cb40 x1874671626740096/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827469 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827444.822931:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626740096/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827469 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827444.822938:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827444.822941:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827444.822944:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626740096:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827444.822955:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626740224:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827444.822973:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827444.822984:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827444/real 1787827444]  req@ffff8cd0c8e9f480 x1874671626740224/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827469 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827444.823000:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626740224/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827469 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827444.823010:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827444.823015:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827444.823019:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626740224:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827445.014313:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827445.398298:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827446.038194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827446.422328:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827447.062213:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827447.446226:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827448.086155:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827448.470246:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827449.110231:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827449.494226:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827449.942219:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827449.942231:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827449.942235:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827449.942246:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1335
00000100:00080000:2.0:1787827449.942258:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827449.942266:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827449.942323:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626740352/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827449.942357:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827449.942361:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827449.942362:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827449.942365:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1335
00000100:00080000:2.0:1787827449.942368:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827449.942389:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827449.942400:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827449.942415:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626740480/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827449.942417:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626740352:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787827449.942433:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827449.942436:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827449.942437:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827449.942448:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1335
00000100:00080000:2.0:1787827449.942452:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827449.942455:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827449.942460:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827449.942469:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626740608/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827449.942477:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827449.942480:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827449.942482:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827449.942484:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1335
00000100:00080000:2.0:1787827449.942487:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827449.942490:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827449.942494:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827449.942501:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626740736/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827449.942543:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626740480:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827449.942569:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626740608:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827449.942586:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626740736:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827450.134212:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827450.518293:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827451.158186:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827451.542216:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827452.182207:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827452.566207:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:1.0:1787827453.014396:0:2344:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:1.0:1787827453.014439:0:2344:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1787827453.014456:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827453.014500:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827453.014525:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827453.014536:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:3.0:1787827453.014553:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827449/real 1787827453]  req@ffff8cd0c3dabc00 x1874671626740480/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827474 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827453.014583:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626740480/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827474 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827453.014596:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827453.014601:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827453.014605:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626740480:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827453.014633:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827449/real 1787827453]  req@ffff8cd0c3daa1c0 x1874671626740608/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827474 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827453.014649:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626740608/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827474 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827453.014657:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827453.014661:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827453.014665:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626740608:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827453.014672:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827449/real 1787827453]  req@ffff8cd0c3dab840 x1874671626740736/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827474 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827453.014684:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626740736/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827474 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827453.014692:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827453.014696:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827453.014699:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626740736:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827453.014705:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827449/real 1787827453]  req@ffff8cd0c3da8000 x1874671626740352/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827474 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827453.014719:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626740352/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827474 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827453.014728:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827453.014731:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827453.014735:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626740352:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787827453.206226:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827453.590242:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827454.230170:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827454.614380:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827455.062186:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827455.062216:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827455.062221:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827455.062230:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1340
00000100:00080000:2.0:1787827455.062243:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827455.062250:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827455.062305:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626740864/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827455.062332:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827455.062336:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827455.062338:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827455.062343:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1340
00000100:00080000:2.0:1787827455.062348:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827455.062354:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827455.062366:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827455.062370:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626740864:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827455.062387:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626740992/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827455.062400:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827455.062403:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827455.062406:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827455.062411:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1340
00000100:00080000:2.0:1787827455.062432:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827455.062438:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827455.062446:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787827455.062450:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827455.062469:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626741120/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827455.062490:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827455.062495:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827455.062500:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1787827455.062502:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827455/real 1787827455]  req@ffff8cd0c3da8780 x1874671626740864/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827480 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827455.062511:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1340
00000100:00080000:2.0:1787827455.062519:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827455.062528:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827455.062536:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827455.062537:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626740864/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827480 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827455.062557:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:2.0:1787827455.062564:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626741248/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827455.062569:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827455.062585:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626740864:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827455.062627:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626740992:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827455.062662:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827455.062677:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827455/real 1787827455]  req@ffff8cd0c3da9a40 x1874671626740992/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827480 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827455.062689:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626740992/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827480 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827455.062698:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827455.062702:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827455.062706:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626740992:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827455.062713:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626741120:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827455.062736:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827455.062764:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827455/real 1787827455]  req@ffff8cd0c3dab0c0 x1874671626741120/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827480 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827455.062778:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626741120/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827480 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827455.062787:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827455.062790:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827455.062792:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626741120:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827455.062801:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626741248:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827455.062822:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827455.062835:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827455/real 1787827455]  req@ffff8cd0c3daa940 x1874671626741248/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827480 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827455.062846:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626741248/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827480 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827455.062854:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827455.062858:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827455.062861:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626741248:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827455.254175:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827455.638215:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827456.278170:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827456.662299:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827457.302203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827457.686171:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827458.326240:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827458.710236:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827459.350191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827459.734204:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827460.182217:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827460.182227:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827460.182231:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827460.182255:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1346
00000100:00080000:3.0:1787827460.182266:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827460.182294:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827460.182333:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626741376/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827460.182364:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827460.182367:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827460.182368:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827460.182371:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1346
00000100:00080000:3.0:1787827460.182375:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827460.182377:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827460.182387:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827460.182397:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626741504/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827460.182407:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827460.182409:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827460.182411:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827460.182416:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1346
00000100:00080000:3.0:1787827460.182422:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1787827460.182424:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626741376:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827460.182428:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827460.182439:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827460.182448:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626741632/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827460.182455:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827460.182458:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827460.182465:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827460.182468:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1346
00000100:00080000:3.0:1787827460.182474:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827460.182478:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827460.182483:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827460.182492:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626741760/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787827460.182511:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827460.182557:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827460/real 1787827460]  req@ffff8cd0c8eac3c0 x1874671626741376/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827485 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827460.182576:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626741376/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827485 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827460.182588:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827460.182593:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827460.182597:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626741376:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827460.182621:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626741504:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827460.182644:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827460.182655:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827460/real 1787827460]  req@ffff8cd0c8ead2c0 x1874671626741504/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827485 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827460.182681:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626741504/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827485 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827460.182690:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827460.182694:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827460.182698:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626741504:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827460.182704:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626741632:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827460.182725:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827460.182735:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827460/real 1787827460]  req@ffff8cd0c8eae1c0 x1874671626741632/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827485 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827460.182748:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626741632/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827485 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827460.182757:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827460.182760:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827460.182764:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626741632:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827460.182770:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626741760:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827460.182786:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827460.182797:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827460/real 1787827460]  req@ffff8cd0c8eae940 x1874671626741760/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827485 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827460.182808:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626741760/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827485 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827460.182829:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827460.182833:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827460.182836:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626741760:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827460.374181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827460.758345:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827461.398246:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827461.782185:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827462.422161:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827462.806210:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827463.446158:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827463.830180:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827464.470153:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827464.854277:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827465.302246:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827465.302260:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827465.302265:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827465.302279:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1351
00000100:00080000:3.0:1787827465.302314:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827465.302327:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827465.302387:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626741888/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827465.302417:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827465.302420:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827465.302422:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827465.302425:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1351
00000100:00080000:3.0:1787827465.302428:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827465.302431:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827465.302454:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827465.302488:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626742016/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827465.302501:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827465.302506:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:2.0:1787827465.302513:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626741888:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827465.302513:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827465.302521:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1351
00000100:00080000:3.0:1787827465.302526:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827465.302542:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827465.302553:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827465.302575:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626742144/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827465.302588:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827465.302592:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827465.302593:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827465.302598:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1351
00000100:00080000:3.0:1787827465.302605:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827465.302609:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827465.302614:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787827465.302618:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827465.302633:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626742272/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827465.302679:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827465/real 1787827465]  req@ffff8cd0c8eac000 x1874671626741888/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827490 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827465.302700:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626741888/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827490 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827465.302714:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827465.302718:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827465.302722:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626741888:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827465.302789:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626742016:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827465.302822:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827465.302849:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827465/real 1787827465]  req@ffff8cd0c8eacf00 x1874671626742016/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827490 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827465.302865:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626742016/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827490 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827465.302876:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827465.302880:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827465.302886:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626742016:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827465.302921:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626742144:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827465.302965:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827465.302989:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827465/real 1787827465]  req@ffff8cd0c8eae940 x1874671626742144/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827490 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827465.303008:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626742144/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827490 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827465.303018:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827465.303022:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827465.303026:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626742144:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827465.303034:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626742272:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827465.303061:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827465.303074:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827465/real 1787827465]  req@ffff8cd0c8eae1c0 x1874671626742272/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827490 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827465.303144:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626742272/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827490 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827465.303155:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827465.303158:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827465.303162:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626742272:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827465.494220:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827465.878195:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827466.518265:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827466.902289:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827467.542158:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827467.926199:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827468.566162:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827468.950258:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827469.590201:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827469.974215:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827470.422218:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827470.422230:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827470.422251:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827470.422261:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1356
00000100:00080000:3.0:1787827470.422273:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827470.422281:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827470.422316:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626742400/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827470.422346:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827470.422350:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827470.422352:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827470.422356:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1356
00000100:00080000:3.0:1787827470.422360:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827470.422365:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827470.422375:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827470.422389:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626742400:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827470.422392:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626742528/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827470.422407:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827470.422411:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827470.422413:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827470.422418:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1356
00000100:00080000:3.0:1787827470.422423:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827470.422427:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827470.422434:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787827470.422447:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827470.422450:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626742656/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827470.422476:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827470.422480:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827470.422482:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827470.422487:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1356
00000100:00080000:3.0:1787827470.422493:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1787827470.422494:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827470/real 1787827470]  req@ffff8cd0c8ead2c0 x1874671626742400/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827495 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827470.422500:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827470.422510:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827470.422526:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626742784/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827470.422546:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626742400/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827495 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827470.422559:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827470.422565:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827470.422570:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626742400:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827470.422596:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626742528:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827470.422622:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827470.422637:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827470/real 1787827470]  req@ffff8cd0c8eac3c0 x1874671626742528/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827495 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827470.422649:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626742528/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827495 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827470.422657:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827470.422661:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827470.422665:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626742528:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827470.422671:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626742656:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827470.422692:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827470.422704:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827470/real 1787827470]  req@ffff8cd0c8eaf840 x1874671626742656/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827495 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827470.422715:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626742656/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827495 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827470.422738:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827470.422742:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827470.422746:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626742656:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827470.422752:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626742784:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827470.422775:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827470.422787:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827470/real 1787827470]  req@ffff8cd0c8eac780 x1874671626742784/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827495 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827470.422800:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626742784/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827495 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827470.422808:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827470.422812:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827470.422815:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626742784:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827470.614196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827470.998260:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827471.638149:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827472.022244:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827472.662178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827473.046260:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827473.686171:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827474.070648:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827474.710239:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827475.094245:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827475.542241:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827475.542253:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827475.542280:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827475.542292:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1361
00000100:00080000:1.0:1787827475.542307:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827475.542316:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827475.542367:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626742912/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827475.542392:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827475.542395:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827475.542397:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827475.542400:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1361
00000100:00080000:1.0:1787827475.542403:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827475.542406:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827475.542417:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827475.542428:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626743040/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827475.542438:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626742912:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787827475.542444:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827475.542448:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827475.542450:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827475.542453:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1361
00000100:00080000:1.0:1787827475.542456:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827475.542458:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827475.542462:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827475.542472:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626743168/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827475.542494:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827475.542497:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827475.542498:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827475.542501:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1361
00000100:00080000:1.0:1787827475.542505:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827475.542508:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827475.542513:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827475.542522:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626743296/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787827475.542529:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827475.542579:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827475/real 1787827475]  req@ffff8cd0d48f4000 x1874671626742912/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827500 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827475.542594:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626742912/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827500 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827475.542603:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827475.542606:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827475.542609:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626742912:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827475.542650:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626743040:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827475.542671:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827475.542681:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827475/real 1787827475]  req@ffff8cd0d48f6580 x1874671626743040/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827500 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827475.542690:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626743040/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827500 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827475.542697:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827475.542699:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827475.542702:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626743040:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827475.542705:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626743168:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827475.542718:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827475.542725:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827475/real 1787827475]  req@ffff8cd0d48f7480 x1874671626743168/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827500 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827475.542732:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626743168/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827500 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827475.542737:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827475.542739:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827475.542741:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626743168:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827475.542745:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626743296:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827475.542755:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827475.542760:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827475/real 1787827475]  req@ffff8cd0d48f4b40 x1874671626743296/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827500 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827475.542781:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626743296/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827500 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827475.542790:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827475.542794:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827475.542797:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626743296:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827475.734193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827476.118302:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827476.758240:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827477.142295:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827477.782191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827478.166224:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827478.806257:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827479.190267:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827479.830182:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827480.214228:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827480.662289:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827480.662300:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827480.662306:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827480.662334:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1366
00000100:00080000:3.0:1787827480.662346:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827480.662353:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827480.662400:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626743424/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827480.662443:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827480.662447:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827480.662468:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827480.662471:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1366
00000100:00080000:3.0:1787827480.662475:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827480.662478:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827480.662505:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827480.662521:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626743552/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827480.662542:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827480.662547:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827480.662549:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827480.662553:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1366
00000100:00080000:3.0:1787827480.662558:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827480.662564:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827480.662571:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827480.662579:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626743424:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827480.662588:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626743680/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827480.662597:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827480.662601:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827480.662603:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827480.662608:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1366
00000100:00080000:3.0:1787827480.662611:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827480.662614:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827480.662617:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827480.662635:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626743808/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787827480.662657:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827480.662703:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827480/real 1787827480]  req@ffff8cd0c8eada40 x1874671626743424/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827505 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827480.662718:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626743424/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827505 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827480.662728:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827480.662731:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827480.662735:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626743424:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827480.662741:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626743552:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827480.662756:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827480.662765:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827480/real 1787827480]  req@ffff8cd0c8eaf0c0 x1874671626743552/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827505 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827480.662774:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626743552/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827505 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827480.662799:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827480.662802:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827480.662804:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626743552:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827480.662828:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626743680:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827480.662845:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827480.662852:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827480/real 1787827480]  req@ffff8cd0c8eac780 x1874671626743680/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827505 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827480.662861:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626743680/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827505 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827480.662867:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827480.662869:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827480.662871:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626743680:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827480.662876:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626743808:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827480.662887:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827480.662894:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827480/real 1787827480]  req@ffff8cd0c8eaf840 x1874671626743808/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827505 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827480.662902:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626743808/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827505 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827480.662907:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827480.662909:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827480.662917:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626743808:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827480.854220:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827481.238223:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827481.878221:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827482.262223:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827482.902193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827483.286204:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827483.926160:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827484.310216:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827484.950175:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827485.334229:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827485.782144:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827485.782154:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827485.782158:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827485.782175:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1371
00000100:00080000:3.0:1787827485.782187:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827485.782194:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827485.782237:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626743936/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827485.782260:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827485.782263:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827485.782265:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827485.782267:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1371
00000100:00080000:3.0:1787827485.782270:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827485.782272:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827485.782283:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827485.782297:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626744064/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827485.782310:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827485.782315:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:2.0:1787827485.782323:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626743936:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827485.782335:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827485.782342:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1371
00000100:00080000:3.0:1787827485.782346:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827485.782351:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827485.782357:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827485.782372:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626744192/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827485.782380:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827485.782383:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827485.782385:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827485.782388:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1371
00000100:00080000:3.0:1787827485.782392:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1787827485.782397:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787827485.782398:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827485.782405:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827485.782415:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626744320/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827485.782444:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827485/real 1787827485]  req@ffff8cd0c8eac3c0 x1874671626743936/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827510 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827485.782459:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626743936/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827510 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827485.782468:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827485.782471:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827485.782474:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626743936:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827485.782494:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626744064:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827485.782510:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827485.782518:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827485/real 1787827485]  req@ffff8cd0c8ead2c0 x1874671626744064/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827510 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827485.782525:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626744064/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827510 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827485.782531:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827485.782533:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827485.782535:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626744064:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827485.782539:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626744192:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827485.782550:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827485.782569:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827485/real 1787827485]  req@ffff8cd0c8eae1c0 x1874671626744192/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827510 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827485.782577:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626744192/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827510 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827485.782583:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827485.782585:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827485.782587:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626744192:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827485.782591:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626744320:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827485.782603:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827485.782609:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827485/real 1787827485]  req@ffff8cd0c8eae940 x1874671626744320/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827510 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827485.782616:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626744320/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827510 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827485.782621:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827485.782623:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827485.782624:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626744320:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827485.974197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827486.358263:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827486.998153:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827487.382285:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827488.022203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827488.406221:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827489.046178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827489.430209:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827490.070176:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827490.454197:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827490.902140:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827490.902148:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827490.902152:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827490.902160:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1376
00000100:00080000:3.0:1787827490.902169:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827490.902174:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827490.902215:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626744448/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827490.902235:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827490.902238:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827490.902239:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827490.902242:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1376
00000100:00080000:3.0:1787827490.902245:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827490.902248:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827490.902256:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827490.902264:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626744576/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827490.902272:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1787827490.902274:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626744448:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827490.902275:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827490.902279:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827490.902285:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1376
00000100:00080000:3.0:1787827490.902287:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827490.902289:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827490.902292:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827490.902301:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626744704/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827490.902308:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827490.902310:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827490.902312:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827490.902319:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1376
00000100:00080000:3.0:1787827490.902335:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827490.902338:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827490.902341:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787827490.902346:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827490.902351:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626744832/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827490.902383:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827490/real 1787827490]  req@ffff8cd0c8eacf00 x1874671626744448/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827515 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827490.902397:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626744448/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827515 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827490.902411:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827490.902414:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827490.902417:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626744448:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827490.902436:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626744576:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827490.902454:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827490.902463:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827490/real 1787827490]  req@ffff8cd0c8eac000 x1874671626744576/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827515 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827490.902482:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626744576/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827515 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827490.902491:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827490.902494:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827490.902498:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626744576:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827490.902504:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626744704:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827490.902526:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827490.902538:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827490/real 1787827490]  req@ffff8cd0c8eae940 x1874671626744704/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827515 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827490.902549:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626744704/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827515 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827490.902555:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827490.902557:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827490.902559:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626744704:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827490.902569:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626744832:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827490.902585:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827490.902593:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827490/real 1787827490]  req@ffff8cd0c8eae1c0 x1874671626744832/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827515 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827490.902601:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626744832/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827515 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827490.902606:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827490.902608:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827490.902610:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626744832:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827491.094233:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827491.478216:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827492.118205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827492.502221:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827493.142159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827493.526325:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827494.166187:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827494.550244:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827495.190187:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827495.574199:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827496.022181:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827496.022189:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827496.022192:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827496.022200:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1381
00000100:00080000:3.0:1787827496.022210:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827496.022216:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827496.022266:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626744960/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827496.022307:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827496.022311:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827496.022313:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827496.022316:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1381
00000100:00080000:3.0:1787827496.022319:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827496.022322:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827496.022331:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827496.022359:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626744960:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827496.022365:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626745088/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827496.022377:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827496.022380:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827496.022382:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827496.022385:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1381
00000100:00080000:3.0:1787827496.022389:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827496.022393:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827496.022398:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827496.022409:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626745216/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827496.022433:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827496.022435:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827496.022436:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827496.022439:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1381
00000100:00080000:3.0:1787827496.022442:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1787827496.022444:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787827496.022448:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827496.022453:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827496.022462:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626745344/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827496.022485:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827496/real 1787827496]  req@ffff8cd0c8ead2c0 x1874671626744960/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827521 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827496.022499:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626744960/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827521 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827496.022507:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827496.022511:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827496.022513:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626744960:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827496.022531:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626745088:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827496.022546:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827496.022555:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827496/real 1787827496]  req@ffff8cd0c8eac3c0 x1874671626745088/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827521 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827496.022572:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626745088/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827521 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827496.022589:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827496.022592:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827496.022594:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626745088:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827496.022597:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626745216:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827496.022612:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827496.022618:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827496/real 1787827496]  req@ffff8cd0c8eaf840 x1874671626745216/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827521 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827496.022626:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626745216/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827521 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827496.022631:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827496.022632:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827496.022634:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626745216:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827496.022638:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626745344:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827496.022648:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827496.022653:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827496/real 1787827496]  req@ffff8cd0c8eac780 x1874671626745344/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827521 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827496.022660:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626745344/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827521 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827496.022674:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827496.022687:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827496.022689:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626745344:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827496.214183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827496.598242:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827497.238296:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827497.622258:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827498.262182:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827498.646193:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827499.286169:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827499.670274:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827500.310196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827500.694187:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827501.142242:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827501.142259:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827501.142306:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827501.142319:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1386
00000100:00080000:3.0:1787827501.142334:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827501.142361:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827501.142433:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626745472/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827501.142494:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827501.142499:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827501.142501:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827501.142507:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1386
00000100:00080000:3.0:1787827501.142512:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827501.142518:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827501.142531:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827501.142554:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626745600/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827501.142566:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626745472:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827501.142591:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827501.142599:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827501.142603:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827501.142612:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1386
00000100:00080000:3.0:1787827501.142622:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827501.142630:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827501.142640:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827501.142682:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626745728/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787827501.142690:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787827501.142713:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827501.142718:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827501.142720:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827501.142726:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1386
00000100:00080000:3.0:1787827501.142760:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827501.142765:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827501.142779:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827501/real 1787827501]  req@ffff8cd0c8eaf0c0 x1874671626745472/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827526 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1787827501.142784:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827501.142809:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626745856/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827501.142826:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626745472/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827526 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827501.142849:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827501.142857:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827501.142865:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626745472:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827501.142905:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626745600:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827501.142950:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827501.142995:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827501/real 1787827501]  req@ffff8cd0c8eada40 x1874671626745600/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827526 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827501.143011:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626745600/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827526 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827501.143021:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827501.143025:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827501.143030:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626745600:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827501.143054:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626745728:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827501.143176:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827501.143196:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827501/real 1787827501]  req@ffff8cd0c8eac780 x1874671626745728/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827526 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827501.143211:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626745728/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827526 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827501.143221:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827501.143227:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827501.143234:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626745728:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827501.143245:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626745856:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827501.143284:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827501.143305:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827501/real 1787827501]  req@ffff8cd0c8eaf840 x1874671626745856/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827526 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827501.143327:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626745856/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827526 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827501.143337:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827501.143340:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827501.143344:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626745856:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827501.334159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827501.718259:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827502.358160:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827502.742169:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827503.382166:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827503.766327:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827504.406191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827504.790239:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827505.430165:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827505.814287:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827506.262235:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827506.262245:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827506.262250:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827506.262262:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1392
00000100:00080000:3.0:1787827506.262277:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827506.262287:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827506.262340:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626745984/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827506.262374:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827506.262378:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827506.262380:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827506.262410:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1392
00000100:00080000:3.0:1787827506.262415:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827506.262421:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827506.262433:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827506.262439:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626745984:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827506.262452:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626746112/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827506.262477:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827506.262481:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827506.262483:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827506.262488:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1392
00000100:00080000:3.0:1787827506.262496:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827506.262501:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827506.262507:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787827506.262520:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827506.262522:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626746240/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827506.262546:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827506.262550:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827506.262552:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827506.262557:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1392
00000100:00080000:3.0:1787827506.262562:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827506.262567:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827506.262573:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827506.262580:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827506/real 1787827506]  req@ffff8cd0c8eac3c0 x1874671626745984/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827531 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827506.262588:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626746368/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827506.262605:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626745984/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827531 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827506.262621:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827506.262625:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827506.262628:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626745984:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827506.262652:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626746112:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827506.262670:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827506.262679:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827506/real 1787827506]  req@ffff8cd0c8ead2c0 x1874671626746112/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827531 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827506.262687:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626746112/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827531 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827506.262693:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827506.262695:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827506.262697:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626746112:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827506.262701:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626746240:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827506.262712:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827506.262718:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827506/real 1787827506]  req@ffff8cd0c8eae1c0 x1874671626746240/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827531 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827506.262726:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626746240/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827531 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827506.262740:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827506.262743:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827506.262745:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626746240:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827506.262749:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626746368:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827506.262763:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827506.262769:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827506/real 1787827506]  req@ffff8cd0c8eae940 x1874671626746368/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827531 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827506.262776:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626746368/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827531 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827506.262782:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827506.262783:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827506.262785:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626746368:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827506.454164:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827506.838220:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827507.478187:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827507.862272:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827508.502219:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827508.886300:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827509.526232:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827509.910252:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827510.550171:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827510.934255:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827511.382182:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827511.382190:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827511.382194:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827511.382202:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1397
00000100:00080000:3.0:1787827511.382212:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827511.382218:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827511.382251:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626746496/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827511.382274:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827511.382282:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827511.382284:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827511.382286:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1397
00000100:00080000:3.0:1787827511.382289:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827511.382292:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827511.382301:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827511.382311:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626746624/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827511.382317:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626746496:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827511.382322:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827511.382326:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827511.382327:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827511.382330:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1397
00000100:00080000:3.0:1787827511.382345:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827511.382349:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827511.382353:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827511.382363:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626746752/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827511.382370:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827511.382373:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827511.382374:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1787827511.382379:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787827511.382382:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1397
00000100:00080000:3.0:1787827511.382386:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827511.382389:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827511.382393:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827511.382401:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626746880/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827511.382418:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827511/real 1787827511]  req@ffff8cd0c8eac000 x1874671626746496/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827536 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827511.382435:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626746496/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827536 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827511.382444:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827511.382447:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827511.382450:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626746496:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827511.382474:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626746624:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827511.382491:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827511.382499:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827511/real 1787827511]  req@ffff8cd0c8eacf00 x1874671626746624/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827536 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827511.382515:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626746624/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827536 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827511.382521:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827511.382524:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827511.382526:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626746624:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827511.382530:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626746752:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827511.382543:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827511.382549:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827511/real 1787827511]  req@ffff8cd0c8eae940 x1874671626746752/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827536 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827511.382557:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626746752/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827536 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827511.382562:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827511.382563:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827511.382565:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626746752:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827511.382569:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626746880:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827511.382579:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827511.382584:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827511/real 1787827511]  req@ffff8cd0c8eae1c0 x1874671626746880/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827536 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827511.382598:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626746880/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827536 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827511.382603:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827511.382605:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827511.382607:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626746880:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827511.574198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827511.958278:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827512.598172:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827512.982390:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827513.622205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827514.006300:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827514.646208:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827515.030279:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827515.670176:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827516.054262:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827516.502233:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827516.502248:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827516.502254:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827516.502267:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1402
00000100:00080000:2.0:1787827516.502284:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827516.502306:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827516.502387:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626747008/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827516.502431:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827516.502437:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827516.502441:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827516.502447:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1402
00000100:00080000:2.0:1787827516.502454:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827516.502460:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827516.502474:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827516.502502:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626747136/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827516.502502:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626747008:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787827516.502565:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827516.502581:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827516.502585:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827516.502609:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1402
00000100:00080000:2.0:1787827516.502618:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827516.502624:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827516.502645:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827516.502670:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626747264/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827516.502683:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626747136:192.168.203.145@tcp:38:kworker.0
00000100:00080000:2.0:1787827516.502700:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827516.502707:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827516.502711:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827516.502721:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1402
00000100:00080000:2.0:1787827516.502730:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827516.502738:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827516.502744:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626747264:192.168.203.145@tcp:8:kworker.0
00000008:00100000:2.0:1787827516.502751:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827516.502779:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab480 x1874671626747392/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827516.502825:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626747392:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827516.694155:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827517.078251:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827517.718239:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827518.102250:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827518.742163:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827519.126296:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:1.0:1787827519.574504:0:2346:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:1.0:1787827519.574572:0:2346:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1787827519.574598:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827519.574685:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827519.574698:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827519.574709:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:3.0:1787827519.574790:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827516/real 1787827519]  req@ffff8cd0c3dab480 x1874671626747392/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827541 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827519.574844:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab480 x1874671626747392/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827541 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827519.574862:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827519.574870:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827519.574876:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626747392:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827519.574887:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827516/real 1787827519]  req@ffff8cd0c3da8f00 x1874671626747264/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827541 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827519.574906:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626747264/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827541 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827519.574917:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827519.574921:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827519.574926:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626747264:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827519.574935:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827516/real 1787827519]  req@ffff8cd0c3da9680 x1874671626747136/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827541 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827519.574957:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626747136/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827541 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827519.574969:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827519.574974:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827519.574978:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626747136:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827519.575013:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827516/real 1787827519]  req@ffff8cd0c3da83c0 x1874671626747008/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827541 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827519.575032:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626747008/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827541 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827519.575042:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827519.575046:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827519.575051:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626747008:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787827519.766241:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827520.150238:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827520.790273:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827521.174297:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827521.622188:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827521.622199:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827521.622204:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827521.622215:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1407
00000100:00080000:2.0:1787827521.622242:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827521.622248:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827521.622280:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8b40 x1874671626747520/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827521.622304:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827521.622306:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827521.622308:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827521.622316:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1407
00000100:00080000:2.0:1787827521.622318:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827521.622321:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827521.622329:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827521.622337:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626747648/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827521.622345:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827521.622348:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827521.622350:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827521.622353:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1407
00000100:00080000:2.0:1787827521.622356:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827521.622360:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827521.622364:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827521.622371:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626747776/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827521.622379:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827521.622388:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827521.622393:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1787827521.622395:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626747520:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787827521.622399:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1407
00000100:00080000:2.0:1787827521.622413:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827521.622421:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827521.622432:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827521.622452:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626747904/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787827521.622491:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827521.622550:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827521/real 1787827521]  req@ffff8cd0c3da8b40 x1874671626747520/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827546 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827521.622573:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8b40 x1874671626747520/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827546 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827521.622587:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827521.622591:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827521.622594:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626747520:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827521.622605:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626747648:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827521.622634:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827521.622649:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827521/real 1787827521]  req@ffff8cd0c3da83c0 x1874671626747648/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827546 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827521.622663:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626747648/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827546 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827521.622690:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827521.622695:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827521.622698:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626747648:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827521.622730:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626747776:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827521.622755:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827521.622767:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827521/real 1787827521]  req@ffff8cd0c3da9680 x1874671626747776/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827546 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827521.622778:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626747776/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827546 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827521.622786:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827521.622790:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827521.622793:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626747776:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827521.622801:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626747904:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827521.622837:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827521.622849:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827521/real 1787827521]  req@ffff8cd0c3da8f00 x1874671626747904/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827546 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827521.622861:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626747904/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827546 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827521.622868:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827521.622871:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827521.622881:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626747904:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827521.814222:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827522.198215:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827522.838246:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827523.222296:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827523.862228:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827524.246210:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827524.886158:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827525.270244:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827525.910173:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827526.294212:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787827526.742190:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827526.742198:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787827526.742202:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827526.742209:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1412
00000100:00080000:0.0:1787827526.742218:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827526.742224:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787827526.742257:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626748032/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827526.742278:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827526.742281:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827526.742282:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827526.742285:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1412
00000100:00080000:0.0:1787827526.742287:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827526.742289:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827526.742297:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827526.742306:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626748160/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827526.742316:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626748032:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787827526.742331:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827526.742334:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787827526.742335:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827526.742339:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1412
00000100:00080000:0.0:1787827526.742342:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827526.742345:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827526.742356:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827526.742367:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626748288/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827526.742374:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827526.742376:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827526.742378:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827526.742380:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1412
00000100:00080000:0.0:1787827526.742384:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1787827526.742387:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787827526.742390:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827526.742395:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827526.742403:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626748416/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827526.742429:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827526/real 1787827526]  req@ffff8cd0c8e9d680 x1874671626748032/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827551 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827526.742443:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626748032/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827551 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827526.742451:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827526.742454:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827526.742457:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626748032:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827526.742476:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626748160:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827526.742493:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827526.742502:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827526/real 1787827526]  req@ffff8cd0c8e9f840 x1874671626748160/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827551 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827526.742510:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626748160/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827551 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827526.742515:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827526.742517:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827526.742519:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626748160:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827526.742531:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626748288:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827526.742546:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827526.742553:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827526/real 1787827526]  req@ffff8cd0c8e9ed00 x1874671626748288/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827551 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827526.742561:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626748288/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827551 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827526.742575:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827526.742577:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827526.742579:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626748288:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827526.742583:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626748416:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827526.742595:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827526.742602:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827526/real 1787827526]  req@ffff8cd0c8e9c3c0 x1874671626748416/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827551 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827526.742609:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626748416/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827551 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827526.742620:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827526.742622:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827526.742624:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626748416:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827526.934226:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827527.318287:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827527.958192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827528.342221:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827528.982287:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827529.366292:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827530.006212:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827530.390271:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827531.030246:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827531.414224:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827531.862235:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827531.862283:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827531.862288:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827531.862300:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1417
00000100:00080000:3.0:1787827531.862313:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827531.862322:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827531.862364:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626748544/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827531.862420:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827531.862425:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827531.862427:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827531.862432:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1417
00000100:00080000:3.0:1787827531.862436:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827531.862441:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827531.862452:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827531.862469:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626748544:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827531.862488:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626748672/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827531.862504:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827531.862507:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827531.862509:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827531.862514:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1417
00000100:00080000:3.0:1787827531.862518:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827531.862523:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827531.862529:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827531.862551:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626748800/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787827531.862556:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787827531.862596:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827531.862600:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827531.862603:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827531.862608:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1417
00000100:00080000:3.0:1787827531.862624:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1787827531.862629:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827531/real 1787827531]  req@ffff8cd0c8ead2c0 x1874671626748544/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827556 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827531.862630:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827531.862638:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827531.862674:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626748544/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827556 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827531.862676:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626748928/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827531.862700:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827531.862711:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827531.862736:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626748544:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827531.862804:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626748672:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827531.862846:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827531.862862:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827531/real 1787827531]  req@ffff8cd0c8eac3c0 x1874671626748672/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827556 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827531.862877:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626748672/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827556 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827531.862899:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827531.862903:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827531.862908:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626748672:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827531.862915:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626748800:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827531.862941:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827531.862954:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827531/real 1787827531]  req@ffff8cd0c8eaf840 x1874671626748800/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827556 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827531.862966:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626748800/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827556 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827531.862992:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827531.862995:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827531.862998:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626748800:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827531.863005:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626748928:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827531.863023:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827531.863034:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827531/real 1787827531]  req@ffff8cd0c8eac780 x1874671626748928/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827556 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827531.863065:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626748928/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827556 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827531.863087:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827531.863091:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827531.863104:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626748928:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827532.054198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827532.438343:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827533.078154:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827533.462238:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827534.102196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827534.486326:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827535.126170:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827535.510296:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827536.150252:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827536.534293:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827536.982191:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827536.982199:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827536.982202:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827536.982210:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1422
00000100:00080000:3.0:1787827536.982221:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827536.982226:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827536.982257:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626749056/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827536.982280:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827536.982283:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827536.982284:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827536.982299:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1422
00000100:00080000:3.0:1787827536.982302:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827536.982305:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827536.982313:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827536.982323:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626749184/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827536.982332:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827536.982337:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:2.0:1787827536.982339:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626749056:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827536.982340:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827536.982345:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1422
00000100:00080000:3.0:1787827536.982349:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827536.982353:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827536.982357:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827536.982366:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626749312/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827536.982373:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827536.982375:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827536.982377:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827536.982379:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1422
00000100:00080000:3.0:1787827536.982381:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827536.982383:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827536.982386:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827536.982393:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626749440/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787827536.982412:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827536.982455:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827536/real 1787827536]  req@ffff8cd0c8eada40 x1874671626749056/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827561 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827536.982474:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626749056/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827561 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827536.982485:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827536.982490:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827536.982494:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626749056:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827536.982516:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626749184:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827536.982539:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827536.982553:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827536/real 1787827536]  req@ffff8cd0c8eaf0c0 x1874671626749184/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827561 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827536.982566:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626749184/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827561 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827536.982575:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827536.982579:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827536.982583:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626749184:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827536.982589:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626749312:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827536.982622:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827536.982643:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827536/real 1787827536]  req@ffff8cd0c8eac780 x1874671626749312/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827561 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827536.982657:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626749312/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827561 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827536.982667:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827536.982670:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827536.982674:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626749312:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827536.982681:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626749440:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827536.982700:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827536.982737:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827536/real 1787827536]  req@ffff8cd0c8eaf840 x1874671626749440/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827561 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827536.982750:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626749440/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827561 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827536.982759:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827536.982762:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827536.982766:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626749440:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827537.174210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827537.558304:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827538.198198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827538.582347:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827539.222235:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827539.606288:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827540.246293:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827540.630296:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827541.270226:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827541.654193:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827542.102182:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827542.102193:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827542.102198:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827542.102209:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1427
00000100:00080000:3.0:1787827542.102222:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827542.102229:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827542.102270:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626749568/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827542.102301:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827542.102305:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827542.102307:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827542.102310:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1427
00000100:00080000:3.0:1787827542.102314:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827542.102319:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827542.102342:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827542.102351:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626749568:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827542.102362:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626749696/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827542.102376:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827542.102380:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827542.102383:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827542.102387:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1427
00000100:00080000:3.0:1787827542.102426:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827542.102432:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827542.102440:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787827542.102446:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827542.102470:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626749824/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827542.102483:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827542.102487:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827542.102489:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827542.102494:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1427
00000100:00080000:3.0:1787827542.102499:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827542.102508:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827542.102511:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827542/real 1787827542]  req@ffff8cd0c8eac3c0 x1874671626749568/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827567 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1787827542.102514:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827542.102533:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626749952/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827542.102548:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626749568/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827567 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827542.102573:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827542.102578:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827542.102582:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626749568:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827542.102609:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626749696:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827542.102638:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827542.102653:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827542/real 1787827542]  req@ffff8cd0c8ead2c0 x1874671626749696/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827567 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827542.102666:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626749696/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827567 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827542.102698:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827542.102702:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827542.102705:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626749696:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827542.102712:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626749824:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827542.102736:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827542.102749:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827542/real 1787827542]  req@ffff8cd0c8eae1c0 x1874671626749824/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827567 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827542.102761:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626749824/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827567 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827542.102769:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827542.102773:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827542.102776:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626749824:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827542.102791:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626749952:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827542.102813:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827542.102824:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827542/real 1787827542]  req@ffff8cd0c8eae940 x1874671626749952/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827567 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827542.102836:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626749952/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827567 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827542.102844:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827542.102847:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827542.102850:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626749952:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827542.294238:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827542.678320:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827543.318190:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827543.702181:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827544.342194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827544.726353:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827545.366228:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827545.750341:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827546.390184:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827546.774244:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827547.223323:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827547.223333:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827547.223337:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827547.223347:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1433
00000100:00080000:1.0:1787827547.223359:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827547.223365:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827547.223401:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626750080/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827547.223429:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827547.223434:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827547.223436:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827547.223439:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1433
00000100:00080000:1.0:1787827547.223442:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827547.223458:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827547.223467:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827547.223474:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626750080:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787827547.223481:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626750208/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827547.223496:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827547.223500:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827547.223502:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827547.223506:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1433
00000100:00080000:1.0:1787827547.223512:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827547.223516:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827547.223523:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787827547.223535:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787827547.223538:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626750336/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827547.223550:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827547.223554:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827547.223556:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827547.223559:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1433
00000100:00080000:1.0:1787827547.223562:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827547.223565:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827547.223570:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827547.223573:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827547/real 1787827547]  req@ffff8cd0d48f4b40 x1874671626750080/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827572 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787827547.223583:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626750464/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827547.223592:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626750080/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827572 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827547.223601:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827547.223604:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827547.223608:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626750080:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827547.223626:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626750208:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827547.223660:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827547.223670:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827547/real 1787827547]  req@ffff8cd0d48f7480 x1874671626750208/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827572 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827547.223697:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626750208/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827572 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827547.223703:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827547.223705:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827547.223708:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626750208:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827547.223712:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626750336:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827547.223727:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827547.223733:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827547/real 1787827547]  req@ffff8cd0d48f6580 x1874671626750336/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827572 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827547.223741:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626750336/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827572 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827547.223746:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827547.223748:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827547.223750:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626750336:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827547.223753:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626750464:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827547.223763:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827547.223769:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827547/real 1787827547]  req@ffff8cd0d48f4000 x1874671626750464/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827572 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827547.223776:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626750464/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827572 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827547.223790:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827547.223792:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827547.223794:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626750464:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827547.414205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827547.798295:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827548.438212:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827548.822256:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827549.462247:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827549.846309:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827550.486202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827550.870350:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827551.510171:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827551.894298:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827552.342162:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827552.342171:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827552.342175:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827552.342184:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1438
00000100:00080000:1.0:1787827552.342194:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827552.342201:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827552.342234:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626750592/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827552.342260:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827552.342265:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827552.342267:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827552.342272:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1438
00000100:00080000:1.0:1787827552.342276:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827552.342281:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827552.342304:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827552.342318:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626750720/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827552.342329:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626750592:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787827552.342331:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827552.342335:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827552.342350:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827552.342354:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1438
00000100:00080000:1.0:1787827552.342358:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827552.342363:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827552.342369:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827552.342380:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626750848/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827552.342391:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:2.0:1787827552.342394:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787827552.342395:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827552.342397:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827552.342400:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1438
00000100:00080000:1.0:1787827552.342404:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827552.342407:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827552.342412:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827552.342423:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626750976/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827552.342434:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827552/real 1787827552]  req@ffff8cd0d48f7840 x1874671626750592/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827577 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827552.342453:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626750592/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827577 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827552.342466:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827552.342471:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827552.342476:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626750592:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827552.342516:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626750720:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827552.342535:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827552.342544:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827552/real 1787827552]  req@ffff8cd0d48f5a40 x1874671626750720/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827577 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827552.342551:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626750720/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827577 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827552.342557:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827552.342559:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827552.342562:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626750720:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827552.342572:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626750848:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827552.342586:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827552.342593:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827552/real 1787827552]  req@ffff8cd0d48f43c0 x1874671626750848/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827577 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827552.342612:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626750848/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827577 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827552.342618:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827552.342621:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827552.342623:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626750848:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827552.342627:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626750976:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827552.342651:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827552.342658:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827552/real 1787827552]  req@ffff8cd0d48f70c0 x1874671626750976/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827577 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827552.342665:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626750976/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827577 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827552.342670:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827552.342671:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827552.342673:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626750976:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827552.534217:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827552.918262:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827553.558198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827553.942275:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827554.582201:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827554.966257:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827555.606209:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827555.990362:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827556.630176:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827557.014253:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827557.462152:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827557.462163:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827557.462185:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827557.462196:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1443
00000100:00080000:1.0:1787827557.462210:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827557.462227:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827557.462280:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626751104/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827557.462314:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827557.462318:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827557.462321:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827557.462325:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1443
00000100:00080000:1.0:1787827557.462329:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827557.462334:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827557.462348:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827557.462365:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626751232/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827557.462374:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827557.462377:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827557.462379:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827557.462382:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1443
00000100:00080000:1.0:1787827557.462386:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827557.462391:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827557.462398:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626751104:192.168.203.145@tcp:250:kworker.0
00000008:00100000:1.0:1787827557.462399:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827557.462419:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626751360/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827557.462442:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827557.462446:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827557.462447:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827557.462452:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1443
00000100:00080000:1.0:1787827557.462458:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827557.462462:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827557.462467:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827557.462477:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626751488/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787827557.462484:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827557.462536:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827557/real 1787827557]  req@ffff8cd0d48f61c0 x1874671626751104/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827582 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827557.462578:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626751104/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827582 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827557.462587:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827557.462591:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827557.462594:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626751104:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827557.462600:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626751232:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827557.462620:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827557.462629:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827557/real 1787827557]  req@ffff8cd0d48f7c00 x1874671626751232/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827582 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827557.462637:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626751232/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827582 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827557.462643:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827557.462645:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827557.462647:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626751232:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827557.462668:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626751360:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827557.462680:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827557.462686:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827557/real 1787827557]  req@ffff8cd0d48f52c0 x1874671626751360/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827582 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827557.462694:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626751360/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827582 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827557.462711:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827557.462713:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827557.462715:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626751360:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827557.462720:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626751488:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827557.462734:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827557.462741:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827557/real 1787827557]  req@ffff8cd0d48f6d00 x1874671626751488/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827582 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827557.462748:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626751488/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827582 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827557.462753:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827557.462755:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827557.462757:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626751488:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827557.654186:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827558.038253:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827558.678164:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827559.062199:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827559.702242:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827560.086276:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827560.726185:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827561.110229:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827561.750228:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827562.134259:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827562.582241:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827562.582257:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827562.582278:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827562.582287:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1448
00000100:00080000:3.0:1787827562.582298:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827562.582304:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827562.582345:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626751616/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827562.582372:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827562.582375:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827562.582377:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827562.582380:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1448
00000100:00080000:3.0:1787827562.582383:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827562.582386:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827562.582395:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827562.582409:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626751744/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827562.582418:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827562.582421:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827562.582423:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827562.582426:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1448
00000100:00080000:3.0:1787827562.582428:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827562.582431:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827562.582436:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827562.582444:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626751616:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827562.582454:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626751872/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827562.582482:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827562.582486:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827562.582488:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827562.582493:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1448
00000100:00080000:3.0:1787827562.582497:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827562.582501:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827562.582506:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827562.582516:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626752000/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787827562.582554:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827562.582620:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827562/real 1787827562]  req@ffff8cd0c8eacf00 x1874671626751616/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827587 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827562.582635:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626751616/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827587 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827562.582646:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827562.582650:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827562.582664:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626751616:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827562.582685:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626751744:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827562.582706:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827562.582716:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827562/real 1787827562]  req@ffff8cd0c8eac000 x1874671626751744/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827587 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827562.582724:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626751744/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827587 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827562.582730:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827562.582733:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827562.582735:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626751744:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827562.582761:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626751872:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827562.582774:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827562.582780:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827562/real 1787827562]  req@ffff8cd0c8eae940 x1874671626751872/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827587 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827562.582788:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626751872/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827587 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827562.582794:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827562.582796:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827562.582798:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626751872:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827562.582802:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626752000:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827562.582813:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827562.582819:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827562/real 1787827562]  req@ffff8cd0c8eae1c0 x1874671626752000/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827587 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827562.582844:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626752000/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827587 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827562.582850:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827562.582852:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827562.582854:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626752000:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827562.774156:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827563.158265:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827563.798184:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827564.182223:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827564.822204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827565.206190:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827565.846216:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827566.230270:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827566.870162:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827567.254315:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827567.702215:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827567.702225:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827567.702229:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827567.702239:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1453
00000100:00080000:3.0:1787827567.702250:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827567.702257:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827567.702296:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626752128/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827567.702323:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827567.702327:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827567.702329:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827567.702332:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1453
00000100:00080000:3.0:1787827567.702335:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827567.702338:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827567.702378:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827567.702387:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626752256/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827567.702394:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626752128:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827567.702399:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827567.702403:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827567.702404:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827567.702408:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1453
00000100:00080000:3.0:1787827567.702411:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827567.702414:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827567.702434:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827567.702447:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626752384/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827567.702457:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827567.702460:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827567.702462:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827567.702465:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1453
00000100:00080000:3.0:1787827567.702469:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827567.702472:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827567.702476:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787827567.702482:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827567.702489:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626752512/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827567.702548:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827567/real 1787827567]  req@ffff8cd0c8ead2c0 x1874671626752128/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827592 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827567.702568:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626752128/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827592 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827567.702579:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827567.702583:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827567.702587:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626752128:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827567.702610:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626752256:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827567.702632:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827567.702644:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827567/real 1787827567]  req@ffff8cd0c8eac3c0 x1874671626752256/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827592 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827567.702655:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626752256/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827592 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827567.702677:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827567.702681:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827567.702684:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626752256:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827567.702700:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626752384:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827567.702722:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827567.702732:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827567/real 1787827567]  req@ffff8cd0c8eaf840 x1874671626752384/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827592 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827567.702743:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626752384/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827592 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827567.702750:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827567.702753:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827567.702756:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626752384:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827567.702760:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626752512:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827567.702774:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827567.702797:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827567/real 1787827567]  req@ffff8cd0c8eac780 x1874671626752512/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827592 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827567.702805:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626752512/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827592 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827567.702810:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827567.702812:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827567.702819:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626752512:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827567.894151:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827568.278242:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827568.918151:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827569.302208:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827569.942175:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827570.326231:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827570.966157:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827571.350288:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827571.990202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827572.374278:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827572.822243:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827572.822255:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827572.822261:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827572.822273:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1458
00000100:00080000:1.0:1787827572.822286:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827572.822295:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827572.822344:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626752640/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827572.822375:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827572.822378:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827572.822379:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827572.822383:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1458
00000100:00080000:1.0:1787827572.822388:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827572.822394:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827572.822405:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827572.822425:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626752640:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787827572.822426:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626752768/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827572.822450:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827572.822455:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827572.822474:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827572.822510:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1458
00000100:00080000:1.0:1787827572.822517:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1787827572.822523:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787827572.822530:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827572.822541:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827572.822557:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626752896/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827572.822567:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827572.822573:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:2.0:1787827572.822575:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827572/real 1787827572]  req@ffff8cd0d48f6d00 x1874671626752640/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827597 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787827572.822576:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827572.822583:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1458
00000100:00080000:1.0:1787827572.822590:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827572.822601:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827572.822609:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827572.822627:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626753024/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827572.822629:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626752640/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827597 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827572.822673:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827572.822681:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827572.822687:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626752640:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827572.822716:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626752768:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827572.822744:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827572.822758:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827572/real 1787827572]  req@ffff8cd0d48f52c0 x1874671626752768/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827597 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827572.822771:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626752768/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827597 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827572.822780:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827572.822783:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827572.822787:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626752768:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827572.822794:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626752896:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827572.822817:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827572.822840:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827572/real 1787827572]  req@ffff8cd0d48f7c00 x1874671626752896/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827597 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827572.822848:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626752896/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827597 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827572.822854:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827572.822856:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827572.822858:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626752896:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827572.822863:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626753024:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827572.822887:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827572.822894:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827572/real 1787827572]  req@ffff8cd0d48f61c0 x1874671626753024/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827597 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827572.822901:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626753024/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827597 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827572.822906:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827572.822908:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827572.822910:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626753024:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827573.014153:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827573.398218:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827574.038207:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827574.422294:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827575.062140:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827575.446268:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827576.086171:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827576.470350:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827577.110172:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827577.494260:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827577.942217:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827577.942228:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827577.942232:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827577.942259:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1463
00000100:00080000:1.0:1787827577.942269:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827577.942277:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827577.942329:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626753152/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827577.942372:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827577.942375:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827577.942376:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827577.942380:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1463
00000100:00080000:1.0:1787827577.942383:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827577.942385:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827577.942394:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827577.942405:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626753280/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827577.942416:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1787827577.942433:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626753152:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787827577.942439:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827577.942444:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827577.942449:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1463
00000100:00080000:1.0:1787827577.942453:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827577.942456:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827577.942461:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827577.942488:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626753408/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827577.942505:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827577.942508:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827577.942510:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827577.942513:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1463
00000100:00080000:1.0:1787827577.942516:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827577.942519:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827577.942525:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787827577.942525:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787827577.942537:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626753536/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827577.942572:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827577/real 1787827577]  req@ffff8cd0d48f70c0 x1874671626753152/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827602 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827577.942592:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626753152/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827602 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827577.942610:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827577.942614:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827577.942627:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626753152:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827577.942652:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626753280:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827577.942676:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827577.942687:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827577/real 1787827577]  req@ffff8cd0d48f43c0 x1874671626753280/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827602 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827577.942696:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626753280/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827602 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827577.942701:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827577.942704:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827577.942706:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626753280:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827577.942710:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626753408:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827577.942723:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827577.942738:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827577/real 1787827577]  req@ffff8cd0d48f5a40 x1874671626753408/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827602 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827577.942746:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626753408/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827602 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827577.942752:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827577.942754:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827577.942757:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626753408:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827577.942766:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626753536:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827577.942781:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827577.942788:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827577/real 1787827577]  req@ffff8cd0d48f7840 x1874671626753536/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827602 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827577.942795:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626753536/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827602 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827577.942800:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827577.942803:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827577.942805:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626753536:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827578.134194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827578.518220:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827579.158154:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827579.542217:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827580.182193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827580.566214:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827581.206202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827581.590202:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827582.230188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827582.614247:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827583.062251:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827583.062301:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827583.062306:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827583.062324:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1468
00000100:00080000:2.0:1787827583.062338:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827583.062348:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827583.062405:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab480 x1874671626753664/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827583.062454:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827583.062460:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827583.062463:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827583.062470:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1468
00000100:00080000:2.0:1787827583.062476:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1787827583.062491:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626753664:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787827583.062523:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827583.062534:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827583.062551:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626753792/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827583.062566:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827583.062571:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827583.062573:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827583.062577:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1468
00000100:00080000:2.0:1787827583.062583:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827583.062588:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827583.062595:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827583.062608:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626753920/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827583.062612:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626753792:192.168.203.145@tcp:38:kworker.0
00000100:00080000:2.0:1787827583.062632:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827583.062636:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827583.062639:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827583.062645:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1468
00000100:00080000:2.0:1787827583.062650:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827583.062655:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827583.062662:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827583.062665:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626753920:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827583.062679:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626754048/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827583.062707:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626754048:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827583.254199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827583.638191:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827584.278216:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827584.662251:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827585.302210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827585.686240:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:1.0:1787827586.134324:0:2343:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:1.0:1787827586.134359:0:2343:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1787827586.134388:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827586.134421:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827586.134427:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827586.134447:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:3.0:1787827586.134485:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827583/real 1787827586]  req@ffff8cd0c3da9a40 x1874671626754048/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827608 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827586.134511:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626754048/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827608 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827586.134522:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827586.134527:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827586.134532:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626754048:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827586.134539:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827583/real 1787827586]  req@ffff8cd0c3dab0c0 x1874671626753920/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827608 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827586.134552:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626753920/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827608 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827586.134568:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827586.134573:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827586.134576:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626753920:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827586.134583:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827583/real 1787827586]  req@ffff8cd0c3daa940 x1874671626753792/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827608 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827586.134595:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626753792/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827608 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827586.134612:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827586.134616:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827586.134620:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626753792:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827586.134626:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827583/real 1787827586]  req@ffff8cd0c3dab480 x1874671626753664/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827608 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827586.134640:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab480 x1874671626753664/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827608 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827586.134648:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827586.134651:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827586.134655:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626753664:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787827586.326206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827586.710332:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827587.350193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827587.734188:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827588.182263:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827588.182272:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827588.182275:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827588.182284:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1474
00000100:00080000:1.0:1787827588.182294:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827588.182301:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827588.182330:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626754176/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827588.182361:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827588.182365:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827588.182366:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827588.182370:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1474
00000100:00080000:1.0:1787827588.182372:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827588.182376:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827588.182384:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827588.182397:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626754304/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827588.182407:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626754176:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787827588.182413:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827588.182418:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827588.182420:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827588.182425:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1474
00000100:00080000:1.0:1787827588.182443:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827588.182448:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827588.182455:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827588.182469:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626754432/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787827588.182478:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787827588.182493:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827588.182496:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827588.182498:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827588.182503:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1474
00000100:00080000:1.0:1787827588.182506:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827588.182510:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827588.182514:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827588.182525:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827588/real 1787827588]  req@ffff8cd0d48f4000 x1874671626754176/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827613 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787827588.182527:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626754560/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827588.182569:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626754176/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827613 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827588.182580:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827588.182585:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827588.182589:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626754176:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827588.182613:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626754304:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827588.182649:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827588.182664:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827588/real 1787827588]  req@ffff8cd0d48f6580 x1874671626754304/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827613 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827588.182677:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626754304/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827613 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827588.182686:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827588.182689:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827588.182693:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626754304:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827588.182699:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626754432:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827588.182721:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827588.182733:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827588/real 1787827588]  req@ffff8cd0d48f7480 x1874671626754432/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827613 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827588.182745:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626754432/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827613 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827588.182753:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827588.182756:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827588.182759:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626754432:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827588.182765:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626754560:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827588.182784:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827588.182795:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827588/real 1787827588]  req@ffff8cd0d48f4b40 x1874671626754560/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827613 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827588.182827:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626754560/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827613 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827588.182836:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827588.182839:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827588.182843:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626754560:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827588.374157:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827588.758278:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827589.398184:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827589.782249:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827590.422230:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827590.806241:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827591.446185:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827591.830185:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827592.470204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827592.854375:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827593.302214:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827593.302239:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827593.302243:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827593.302252:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1479
00000100:00080000:2.0:1787827593.302263:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827593.302270:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827593.302304:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626754688/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827593.302356:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827593.302359:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827593.302361:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827593.302365:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1479
00000100:00080000:2.0:1787827593.302369:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827593.302373:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827593.302383:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827593.302396:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626754816/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827593.302407:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827593.302411:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827593.302413:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827593.302417:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1479
00000100:00080000:2.0:1787827593.302420:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827593.302422:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827593.302427:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827593.302432:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626754688:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827593.302440:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626754944/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827593.302459:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827593.302463:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827593.302465:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827593.302480:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1479
00000100:00080000:2.0:1787827593.302485:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827593.302489:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827593.302494:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827593.302507:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626755072/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787827593.302518:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827593.302562:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827593/real 1787827593]  req@ffff8cd0c3da8780 x1874671626754688/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827618 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827593.302580:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626754688/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827618 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827593.302590:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827593.302594:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827593.302598:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626754688:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827593.302605:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626754816:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827593.302624:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827593.302635:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827593/real 1787827593]  req@ffff8cd0c3da8000 x1874671626754816/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827618 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827593.302646:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626754816/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827618 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827593.302678:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827593.302681:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827593.302684:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626754816:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827593.302706:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626754944:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827593.302727:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827593.302736:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827593/real 1787827593]  req@ffff8cd0c3dab840 x1874671626754944/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827618 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827593.302747:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626754944/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827618 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827593.302754:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827593.302757:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827593.302760:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626754944:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827593.302765:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626755072:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827593.302778:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827593.302786:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827593/real 1787827593]  req@ffff8cd0c3daa1c0 x1874671626755072/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827618 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827593.302795:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626755072/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827618 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827593.302802:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827593.302804:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827593.302822:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626755072:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827593.494296:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827593.878210:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827594.518169:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827594.902326:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827595.542196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827595.926171:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827596.566215:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827596.950430:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827597.590192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827597.974229:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827598.422174:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827598.422184:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827598.422188:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827598.422199:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1484
00000100:00080000:2.0:1787827598.422211:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827598.422219:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827598.422261:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626755200/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827598.422337:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827598.422339:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827598.422341:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827598.422344:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1484
00000100:00080000:2.0:1787827598.422346:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827598.422350:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827598.422372:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827598.422384:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626755328/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827598.422388:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626755200:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787827598.422399:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827598.422402:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827598.422404:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827598.422407:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1484
00000100:00080000:2.0:1787827598.422411:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827598.422413:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827598.422417:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827598.422427:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626755456/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827598.422444:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827598.422446:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827598.422447:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827598.422450:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1484
00000100:00080000:2.0:1787827598.422452:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827598.422455:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827598.422457:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787827598.422463:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827598.422467:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626755584/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827598.422502:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827598/real 1787827598]  req@ffff8cd0c3dabc00 x1874671626755200/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827623 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827598.422524:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626755200/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827623 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827598.422532:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827598.422535:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827598.422538:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626755200:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827598.422557:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626755328:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827598.422572:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827598.422580:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827598/real 1787827598]  req@ffff8cd0c3da9e00 x1874671626755328/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827623 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827598.422588:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626755328/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827623 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827598.422593:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827598.422595:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827598.422597:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626755328:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827598.422601:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626755456:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827598.422612:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827598.422627:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827598/real 1787827598]  req@ffff8cd0c3da92c0 x1874671626755456/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827623 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827598.422635:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626755456/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827623 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827598.422640:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827598.422642:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827598.422644:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626755456:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827598.422648:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626755584:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827598.422659:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827598.422665:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827598/real 1787827598]  req@ffff8cd0c3daa580 x1874671626755584/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827623 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827598.422672:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626755584/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827623 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827598.422695:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827598.422698:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827598.422701:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626755584:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827598.614210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827598.998336:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827599.638215:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827600.022309:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827600.662276:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827601.046274:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827601.686221:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827602.070188:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827602.710229:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827603.094377:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787827603.542225:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827603.542238:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787827603.542258:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827603.542270:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1489
00000100:00080000:0.0:1787827603.542293:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827603.542302:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787827603.542343:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626755712/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827603.542373:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827603.542376:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827603.542378:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827603.542381:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1489
00000100:00080000:0.0:1787827603.542385:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827603.542403:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827603.542413:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827603.542438:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626755840/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827603.542448:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626755712:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787827603.542452:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827603.542460:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787827603.542461:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827603.542465:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1489
00000100:00080000:0.0:1787827603.542468:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827603.542471:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827603.542476:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827603.542496:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626755968/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827603.542513:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827603.542516:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827603.542518:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827603.542523:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1489
00000100:00080000:0.0:1787827603.542526:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827603.542529:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827603.542534:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787827603.542540:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827603.542548:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626756096/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827603.542595:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827603/real 1787827603]  req@ffff8cd0c8e9cf00 x1874671626755712/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827628 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827603.542617:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626755712/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827628 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827603.542641:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827603.542647:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827603.542652:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626755712:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827603.542683:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626755840:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827603.542714:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827603.542730:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827603/real 1787827603]  req@ffff8cd0c8e9e940 x1874671626755840/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827628 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827603.542746:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626755840/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827628 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827603.542758:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827603.542762:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827603.542767:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626755840:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827603.542775:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626755968:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827603.542809:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827603.542822:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827603/real 1787827603]  req@ffff8cd0c8e9e580 x1874671626755968/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827628 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827603.542837:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626755968/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827628 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827603.542857:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827603.542862:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827603.542866:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626755968:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827603.542881:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626756096:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827603.542906:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827603.542919:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827603/real 1787827603]  req@ffff8cd0c8e9de00 x1874671626756096/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827628 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827603.542935:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626756096/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827628 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827603.542944:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827603.542949:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827603.542953:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626756096:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827603.734157:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827604.118159:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827604.758210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827605.142355:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827605.782211:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827606.166322:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827606.806250:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827607.190231:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827607.830176:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827608.214209:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827608.662348:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827608.662362:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827608.662369:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827608.662381:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1494
00000100:00080000:3.0:1787827608.662396:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827608.662405:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827608.662454:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626756224/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827608.662498:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827608.662515:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827608.662518:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827608.662523:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1494
00000100:00080000:3.0:1787827608.662528:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827608.662555:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827608.662569:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827608.662603:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626756352/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827608.662618:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827608.662622:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827608.662625:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827608.662630:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1494
00000100:00080000:3.0:1787827608.662636:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1787827608.662643:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626756224:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827608.662662:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827608.662673:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827608.662687:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626756480/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827608.662704:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827608.662710:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827608.662713:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827608.662719:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1494
00000100:00080000:3.0:1787827608.662738:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827608.662746:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827608.662762:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787827608.662767:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827608.662787:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626756608/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827608.662820:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827608/real 1787827608]  req@ffff8cd0c8eaf0c0 x1874671626756224/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827633 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827608.662854:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626756224/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827633 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827608.662862:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827608.662865:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827608.662868:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626756224:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827608.662874:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626756352:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827608.662890:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827608.662898:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827608/real 1787827608]  req@ffff8cd0c8eada40 x1874671626756352/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827633 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827608.662906:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626756352/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827633 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827608.662911:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827608.662913:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827608.662915:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626756352:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827608.662935:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626756480:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827608.662947:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827608.662953:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827608/real 1787827608]  req@ffff8cd0c8eac780 x1874671626756480/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827633 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827608.662974:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626756480/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827633 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827608.662980:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827608.662983:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827608.662985:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626756480:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827608.662989:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626756608:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827608.663002:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827608.663009:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827608/real 1787827608]  req@ffff8cd0c8eaf840 x1874671626756608/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827633 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827608.663017:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626756608/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827633 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827608.663023:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827608.663024:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827608.663026:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626756608:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827608.854284:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827609.238321:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827609.878154:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827610.262269:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827610.902188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827611.286264:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827611.926200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827612.310245:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827612.950263:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827613.334360:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827613.782208:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827613.782240:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827613.782246:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827613.782259:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1499
00000100:00080000:3.0:1787827613.782285:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827613.782295:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827613.782338:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626756736/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827613.782370:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827613.782374:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827613.782376:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827613.782380:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1499
00000100:00080000:3.0:1787827613.782384:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827613.782388:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827613.782398:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827613.782410:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626756864/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827613.782421:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827613.782425:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827613.782426:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827613.782429:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1499
00000100:00080000:3.0:1787827613.782432:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1787827613.782438:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626756736:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827613.782440:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827613.782449:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827613.782459:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626756992/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827613.782481:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827613.782484:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827613.782486:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827613.782491:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1499
00000100:00080000:3.0:1787827613.782506:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827613.782511:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827613.782516:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787827613.782524:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827613.782533:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626757120/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827613.782586:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827613/real 1787827613]  req@ffff8cd0c8eac3c0 x1874671626756736/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827638 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827613.782624:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626756736/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827638 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827613.782651:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827613.782655:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827613.782660:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626756736:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827613.782667:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626756864:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827613.782694:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827613.782715:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827613/real 1787827613]  req@ffff8cd0c8ead2c0 x1874671626756864/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827638 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827613.782727:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626756864/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827638 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827613.782735:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827613.782738:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827613.782741:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626756864:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827613.782764:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626756992:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827613.782780:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827613.782795:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827613/real 1787827613]  req@ffff8cd0c8eae1c0 x1874671626756992/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827638 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827613.782808:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626756992/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827638 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827613.782820:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827613.782824:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827613.782829:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626756992:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827613.782848:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626757120:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827613.782878:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827613.782894:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827613/real 1787827613]  req@ffff8cd0c8eae940 x1874671626757120/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827638 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827613.782906:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626757120/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827638 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827613.782913:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827613.782916:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827613.782919:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626757120:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827613.974159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827614.358248:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827614.998256:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827615.382364:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827616.022224:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827616.406287:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827617.046197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827617.430254:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827618.070157:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827618.454202:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827618.902209:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827618.902221:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827618.902226:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827618.902240:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1504
00000100:00080000:3.0:1787827618.902267:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827618.902278:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827618.902333:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626757248/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827618.902385:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827618.902390:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827618.902392:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827618.902397:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1504
00000100:00080000:3.0:1787827618.902402:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827618.902407:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827618.902450:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827618.902478:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626757248:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827618.902485:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626757376/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827618.902498:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827618.902501:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827618.902503:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827618.902506:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1504
00000100:00080000:3.0:1787827618.902510:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827618.902514:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827618.902519:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827618.902552:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626757504/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827618.902564:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827618.902569:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827618.902582:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827618.902589:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1504
00000100:00080000:3.0:1787827618.902593:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1787827618.902600:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787827618.902601:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827618.902612:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827618.902627:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626757632/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827618.902664:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827618/real 1787827618]  req@ffff8cd0c8eac000 x1874671626757248/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827643 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827618.902689:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626757248/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827643 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827618.902703:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827618.902707:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827618.902712:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626757248:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827618.902741:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626757376:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827618.902770:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827618.902785:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827618/real 1787827618]  req@ffff8cd0c8eacf00 x1874671626757376/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827643 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827618.902829:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626757376/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827643 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827618.902840:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827618.902844:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827618.902848:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626757376:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827618.902866:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626757504:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827618.902924:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827618.902947:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827618/real 1787827618]  req@ffff8cd0c8eae940 x1874671626757504/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827643 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827618.902972:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626757504/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827643 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827618.902988:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827618.902993:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827618.903000:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626757504:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827618.903011:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626757632:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827618.903049:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827618.903066:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827618/real 1787827618]  req@ffff8cd0c8eae1c0 x1874671626757632/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827643 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827618.903081:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626757632/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827643 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827618.903176:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827618.903182:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827618.903186:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626757632:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827619.094206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827619.478282:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827620.118193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827620.502203:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827621.142199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827621.526230:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827622.166222:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827622.550314:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827623.190206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827623.574344:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827624.022167:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827624.022175:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827624.022192:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827624.022213:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1509
00000100:00080000:3.0:1787827624.022223:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827624.022230:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827624.022261:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626757760/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827624.022285:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827624.022288:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827624.022289:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827624.022292:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1509
00000100:00080000:3.0:1787827624.022295:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827624.022298:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827624.022305:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827624.022315:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626757888/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827624.022324:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827624.022327:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827624.022328:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827624.022331:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1509
00000100:00080000:3.0:1787827624.022335:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827624.022339:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827624.022344:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827624.022362:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626758016/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827624.022381:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827624.022384:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827624.022386:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827624.022394:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1509
00000100:00100000:2.0:1787827624.022397:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626757760:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827624.022399:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827624.022410:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827624.022416:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827624.022428:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626758144/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787827624.022489:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827624.022539:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827624/real 1787827624]  req@ffff8cd0c8ead2c0 x1874671626757760/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827649 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827624.022555:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626757760/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827649 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827624.022566:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827624.022570:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827624.022573:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626757760:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827624.022580:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626757888:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827624.022598:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827624.022608:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827624/real 1787827624]  req@ffff8cd0c8eac3c0 x1874671626757888/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827649 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827624.022618:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626757888/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827649 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827624.022642:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827624.022645:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827624.022648:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626757888:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827624.022694:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626758016:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827624.022716:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827624.022727:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827624/real 1787827624]  req@ffff8cd0c8eaf840 x1874671626758016/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827649 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827624.022738:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626758016/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827649 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827624.022746:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827624.022750:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827624.022753:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626758016:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827624.022759:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626758144:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827624.022773:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827624.022781:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827624/real 1787827624]  req@ffff8cd0c8eac780 x1874671626758144/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827649 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827624.022792:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626758144/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827649 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827624.022799:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827624.022801:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827624.022804:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626758144:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827624.214209:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827624.598239:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827625.238189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827625.622308:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827626.262228:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827626.646258:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827627.286257:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827627.670264:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827628.310222:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827628.694261:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827629.142156:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827629.142164:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827629.142168:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827629.142176:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1514
00000100:00080000:3.0:1787827629.142186:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827629.142192:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827629.142224:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626758272/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827629.142259:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827629.142261:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827629.142263:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827629.142265:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1514
00000100:00080000:3.0:1787827629.142268:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827629.142270:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827629.142291:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827629.142300:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626758400/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827629.142308:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827629.142310:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827629.142311:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827629.142313:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1514
00000100:00080000:3.0:1787827629.142316:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827629.142326:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827629.142330:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827629.142342:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626758528/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827629.142352:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827629.142355:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827629.142356:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827629.142359:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1514
00000100:00080000:3.0:1787827629.142362:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827629.142367:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827629.142371:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827629.142377:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626758272:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827629.142382:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626758656/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787827629.142490:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827629.142550:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827629/real 1787827629]  req@ffff8cd0c8eada40 x1874671626758272/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827654 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827629.142590:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626758272/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827654 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827629.142603:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827629.142608:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827629.142613:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626758272:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827629.142621:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626758400:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827629.142648:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827629.142661:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827629/real 1787827629]  req@ffff8cd0c8eaf0c0 x1874671626758400/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827654 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827629.142674:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626758400/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827654 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827629.142682:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827629.142685:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827629.142688:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626758400:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827629.142721:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626758528:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827629.142740:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827629.142751:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827629/real 1787827629]  req@ffff8cd0c8eac780 x1874671626758528/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827654 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827629.142762:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626758528/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827654 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827629.142784:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827629.142788:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827629.142792:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626758528:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827629.142798:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626758656:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827629.142820:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827629.142830:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827629/real 1787827629]  req@ffff8cd0c8eaf840 x1874671626758656/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827654 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827629.142841:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626758656/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827654 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827629.142849:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827629.142852:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827629.142855:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626758656:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827629.334154:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827629.718340:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827630.358194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827630.742212:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827631.382183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827631.766328:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827632.406283:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827632.790292:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827633.430188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827633.814338:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827634.262176:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827634.262185:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827634.262206:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827634.262215:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1520
00000100:00080000:3.0:1787827634.262226:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827634.262232:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827634.262282:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626758784/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827634.262307:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827634.262310:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827634.262312:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827634.262314:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1520
00000100:00080000:3.0:1787827634.262317:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827634.262320:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827634.262328:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827634.262338:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626758912/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827634.262346:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827634.262366:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827634.262369:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827634.262372:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1520
00000100:00080000:3.0:1787827634.262374:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827634.262378:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827634.262384:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827634.262391:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626759040/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827634.262414:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827634.262417:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827634.262419:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1787827634.262424:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626758784:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827634.262425:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1520
00000100:00080000:3.0:1787827634.262438:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827634.262443:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827634.262448:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827634.262478:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626759168/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787827634.262536:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827634.262585:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827634/real 1787827634]  req@ffff8cd0c8eac3c0 x1874671626758784/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827659 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827634.262601:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626758784/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827659 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827634.262611:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827634.262614:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827634.262617:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626758784:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827634.262637:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626758912:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827634.262657:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827634.262666:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827634/real 1787827634]  req@ffff8cd0c8ead2c0 x1874671626758912/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827659 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827634.262674:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626758912/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827659 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827634.262681:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827634.262683:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827634.262685:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626758912:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827634.262711:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626759040:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827634.262723:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827634.262738:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827634/real 1787827634]  req@ffff8cd0c8eae1c0 x1874671626759040/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827659 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827634.262746:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626759040/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827659 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827634.262752:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827634.262754:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827634.262756:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626759040:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827634.262760:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626759168:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827634.262770:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827634.262776:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827634/real 1787827634]  req@ffff8cd0c8eae940 x1874671626759168/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827659 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827634.262790:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626759168/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827659 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827634.262806:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827634.262808:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827634.262810:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626759168:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827634.454194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827634.838166:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827635.478133:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827635.862251:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827636.502171:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827636.886164:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827637.526197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827637.910304:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827638.550181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827638.934176:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827639.382168:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827639.382189:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827639.382192:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827639.382201:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1525
00000100:00080000:3.0:1787827639.382211:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827639.382217:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827639.382266:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626759296/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827639.382289:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827639.382309:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827639.382310:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827639.382313:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1525
00000100:00080000:3.0:1787827639.382316:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827639.382319:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827639.382342:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827639.382354:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626759424/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827639.382380:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1787827639.382385:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626759296:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827639.382385:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827639.382389:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827639.382396:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1525
00000100:00080000:3.0:1787827639.382399:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827639.382402:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827639.382406:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827639.382415:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626759552/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827639.382422:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827639.382425:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827639.382426:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827639.382429:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1525
00000100:00080000:3.0:1787827639.382459:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827639.382463:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827639.382467:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787827639.382476:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827639.382478:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626759680/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827639.382519:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827639/real 1787827639]  req@ffff8cd0c8eacf00 x1874671626759296/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827664 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827639.382536:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626759296/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827664 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827639.382545:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827639.382548:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827639.382552:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626759296:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827639.382572:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626759424:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827639.382591:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827639.382600:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827639/real 1787827639]  req@ffff8cd0c8eac000 x1874671626759424/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827664 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827639.382609:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626759424/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827664 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827639.382629:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827639.382632:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827639.382634:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626759424:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827639.382639:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626759552:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827639.382656:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827639.382679:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827639/real 1787827639]  req@ffff8cd0c8eae940 x1874671626759552/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827664 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827639.382687:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626759552/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827664 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827639.382692:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827639.382694:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827639.382696:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626759552:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827639.382700:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626759680:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827639.382711:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827639.382717:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827639/real 1787827639]  req@ffff8cd0c8eae1c0 x1874671626759680/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827664 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827639.382724:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626759680/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827664 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827639.382729:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827639.382731:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827639.382737:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626759680:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827639.574149:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827639.958268:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827640.598159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827640.982228:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827641.622198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827642.006238:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827642.646198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827643.030260:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827643.670192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827644.054220:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787827644.502196:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827644.502205:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787827644.502208:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827644.502217:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1530
00000100:00080000:0.0:1787827644.502228:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827644.502235:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787827644.502276:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626759808/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827644.502302:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827644.502305:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827644.502306:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827644.502309:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1530
00000100:00080000:0.0:1787827644.502327:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827644.502331:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827644.502340:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827644.502344:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626759808:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827644.502355:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626759936/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827644.502365:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827644.502368:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787827644.502369:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827644.502373:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1530
00000100:00080000:0.0:1787827644.502375:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827644.502377:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827644.502381:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827644.502389:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626760064/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827644.502396:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827644.502398:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827644.502400:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1787827644.502404:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787827644.502405:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1530
00000100:00080000:0.0:1787827644.502411:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827644.502414:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827644.502418:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827644.502427:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626760192/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827644.502447:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827644/real 1787827644]  req@ffff8cd0c8e9fc00 x1874671626759808/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827669 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827644.502464:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626759808/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827669 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827644.502473:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827644.502476:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827644.502479:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626759808:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827644.502496:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626759936:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827644.502511:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827644.502519:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827644/real 1787827644]  req@ffff8cd0c8e9de00 x1874671626759936/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827669 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827644.502527:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626759936/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827669 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827644.502533:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827644.502535:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827644.502546:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626759936:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827644.502551:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626760064:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827644.502563:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827644.502579:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827644/real 1787827644]  req@ffff8cd0c8e9e580 x1874671626760064/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827669 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827644.502588:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626760064/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827669 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827644.502593:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827644.502595:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827644.502597:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626760064:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827644.502601:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626760192:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827644.502613:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827644.502619:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827644/real 1787827644]  req@ffff8cd0c8e9e940 x1874671626760192/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827669 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827644.502626:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626760192/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827669 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827644.502631:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827644.502633:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827644.502635:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626760192:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827644.694148:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827645.078244:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827645.718159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827646.102256:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827646.742151:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827647.126296:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827647.766171:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827648.150196:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827648.790164:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827649.174238:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827649.622179:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827649.622187:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827649.622190:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827649.622199:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1535
00000100:00080000:3.0:1787827649.622208:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827649.622213:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827649.622246:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626760320/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827649.622270:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827649.622273:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827649.622274:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827649.622277:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1535
00000100:00080000:3.0:1787827649.622279:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827649.622281:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827649.622289:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827649.622298:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626760448/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827649.622305:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827649.622308:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827649.622309:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827649.622311:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1535
00000100:00080000:3.0:1787827649.622313:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1787827649.622319:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626760320:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827649.622331:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827649.622337:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827649.622346:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626760576/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827649.622361:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827649.622363:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827649.622365:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827649.622368:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1535
00000100:00080000:3.0:1787827649.622372:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827649.622374:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827649.622377:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827649.622384:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626760704/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827649.622408:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626760448:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827649.622436:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626760576:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827649.622451:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626760704:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827649.814147:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827650.198262:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827650.838155:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827651.222255:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827651.862179:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827652.246265:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:1.0:1787827652.694363:0:2345:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:1.0:1787827652.694406:0:2345:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1787827652.694427:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827652.694459:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827652.694478:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827652.694482:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:2.0:1787827652.694535:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827649/real 1787827652]  req@ffff8cd0c8eaf840 x1874671626760576/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827674 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827652.694557:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626760576/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827674 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827652.694566:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827652.694570:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827652.694573:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626760576:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827652.694578:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827649/real 1787827652]  req@ffff8cd0c8eac780 x1874671626760704/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827674 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827652.694586:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626760704/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827674 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827652.694591:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827652.694593:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827652.694595:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626760704:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827652.694599:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827649/real 1787827652]  req@ffff8cd0c8ead2c0 x1874671626760320/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827674 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827652.694607:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626760320/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827674 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827652.694611:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827652.694613:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827652.694615:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626760320:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827652.694632:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827649/real 1787827652]  req@ffff8cd0c8eac3c0 x1874671626760448/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827674 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827652.694641:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626760448/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827674 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827652.694647:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827652.694651:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827652.694654:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626760448:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827652.886140:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827653.270298:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827653.910165:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827654.294226:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827654.742164:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827654.742175:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827654.742179:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827654.742189:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1540
00000100:00080000:3.0:1787827654.742201:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827654.742208:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827654.742244:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626760832/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827654.742272:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827654.742288:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827654.742291:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827654.742296:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1540
00000100:00080000:3.0:1787827654.742301:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827654.742307:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827654.742318:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626760832:192.168.203.145@tcp:250:kworker.0
00000008:00100000:3.0:1787827654.742318:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827654.742335:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626760960/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827654.742349:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827654.742353:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827654.742355:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827654.742360:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1540
00000100:00080000:3.0:1787827654.742375:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827654.742380:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827654.742386:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787827654.742401:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827654.742402:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626761088/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827654.742415:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827654.742419:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827654.742421:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827654.742426:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1540
00000100:00080000:3.0:1787827654.742431:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827654.742435:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827654.742447:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827654.742454:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827654/real 1787827654]  req@ffff8cd0c8eaf0c0 x1874671626760832/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827679 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827654.742463:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626761216/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827654.742483:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626760832/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827679 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827654.742496:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827654.742501:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827654.742505:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626760832:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827654.742529:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626760960:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827654.742559:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827654.742573:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827654/real 1787827654]  req@ffff8cd0c8eada40 x1874671626760960/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827679 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827654.742586:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626760960/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827679 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827654.742602:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827654.742606:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827654.742609:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626760960:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827654.742615:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626761088:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827654.742640:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827654.742652:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827654/real 1787827654]  req@ffff8cd0c8eac3c0 x1874671626761088/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827679 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827654.742664:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626761088/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827679 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827654.742672:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827654.742675:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827654.742678:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626761088:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827654.742685:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626761216:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827654.742704:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827654.742714:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827654/real 1787827654]  req@ffff8cd0c8ead2c0 x1874671626761216/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827679 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827654.742725:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626761216/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827679 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827654.742732:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827654.742735:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827654.742749:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626761216:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827654.934149:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827655.318259:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827655.958156:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827656.342228:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827656.982132:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827657.366222:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827658.006188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827658.390201:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827659.030228:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827659.414241:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827659.862245:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827659.862253:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827659.862257:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827659.862265:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1545
00000100:00080000:1.0:1787827659.862274:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827659.862279:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827659.862311:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626761344/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827659.862337:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827659.862339:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827659.862341:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827659.862344:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1545
00000100:00080000:1.0:1787827659.862346:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827659.862348:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827659.862355:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827659.862371:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626761472/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827659.862380:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827659.862383:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:2.0:1787827659.862383:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626761344:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787827659.862387:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827659.862393:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1545
00000100:00080000:1.0:1787827659.862410:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827659.862414:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827659.862419:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827659.862429:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626761600/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827659.862436:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827659.862439:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827659.862440:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1787827659.862444:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787827659.862448:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1545
00000100:00080000:1.0:1787827659.862452:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827659.862455:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827659.862459:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827659.862467:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626761728/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827659.862500:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827659/real 1787827659]  req@ffff8cd0d48f4b40 x1874671626761344/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827684 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827659.862515:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626761344/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827684 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827659.862523:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827659.862526:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827659.862528:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626761344:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827659.862547:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626761472:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827659.862563:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827659.862571:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827659/real 1787827659]  req@ffff8cd0d48f7480 x1874671626761472/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827684 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827659.862579:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626761472/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827684 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827659.862585:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827659.862587:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827659.862589:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626761472:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827659.862592:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626761600:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827659.862603:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827659.862619:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827659/real 1787827659]  req@ffff8cd0d48f6580 x1874671626761600/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827684 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827659.862626:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626761600/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827684 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827659.862632:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827659.862634:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827659.862636:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626761600:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827659.862639:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626761728:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827659.862651:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827659.862657:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827659/real 1787827659]  req@ffff8cd0d48f4000 x1874671626761728/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827684 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827659.862664:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626761728/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827684 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827659.862669:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827659.862670:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827659.862672:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626761728:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827660.054205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827660.438226:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827661.078193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827661.462273:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827662.102181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827662.486244:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827663.126196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827663.510208:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827664.150169:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827664.534229:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827664.982207:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827664.982217:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827664.982221:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827664.982231:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1550
00000100:00080000:3.0:1787827664.982252:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827664.982260:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827664.982295:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626761856/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827664.982318:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827664.982320:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827664.982322:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827664.982325:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1550
00000100:00080000:3.0:1787827664.982328:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827664.982330:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827664.982344:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827664.982355:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626761984/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827664.982365:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626761856:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827664.982365:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827664.982383:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827664.982384:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827664.982388:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1550
00000100:00080000:3.0:1787827664.982393:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827664.982396:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827664.982401:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827664.982411:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626762112/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827664.982419:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827664.982421:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000800:00000100:2.0:1787827664.982424:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787827664.982426:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827664.982432:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1550
00000100:00080000:3.0:1787827664.982436:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827664.982442:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827664.982447:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827664.982461:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626762240/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827664.982476:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827664/real 1787827664]  req@ffff8cd0c8eac780 x1874671626761856/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827689 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827664.982492:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626761856/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827689 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827664.982507:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827664.982510:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827664.982513:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626761856:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827664.982532:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626761984:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827664.982554:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827664.982568:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827664/real 1787827664]  req@ffff8cd0c8eaf840 x1874671626761984/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827689 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827664.982582:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626761984/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827689 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827664.982592:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827664.982595:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827664.982598:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626761984:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827664.982603:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626762112:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827664.982625:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827664.982638:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827664/real 1787827664]  req@ffff8cd0c8eae1c0 x1874671626762112/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827689 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827664.982651:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626762112/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827689 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827664.982660:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827664.982662:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827664.982664:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626762112:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827664.982674:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626762240:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827664.982691:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827664.982699:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827664/real 1787827664]  req@ffff8cd0c8eae940 x1874671626762240/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827689 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827664.982707:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626762240/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827689 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827664.982712:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827664.982714:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827664.982716:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626762240:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827665.174231:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827665.558267:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827666.198174:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827666.582259:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827667.222199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827667.606275:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827668.246161:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827668.630201:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827669.270191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827669.654243:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827670.102165:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827670.102175:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827670.102178:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827670.102186:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1555
00000100:00080000:3.0:1787827670.102197:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827670.102203:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827670.102233:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626762368/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827670.102267:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827670.102269:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827670.102271:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827670.102274:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1555
00000100:00080000:3.0:1787827670.102277:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827670.102279:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827670.102287:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827670.102297:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626762496/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827670.102305:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827670.102307:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827670.102309:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827670.102313:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1555
00000100:00100000:2.0:1787827670.102315:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626762368:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827670.102316:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827670.102323:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827670.102327:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827670.102335:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626762624/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827670.102342:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827670.102345:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827670.102347:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827670.102349:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1555
00000100:00080000:3.0:1787827670.102364:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827670.102368:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827670.102372:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787827670.102376:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827670.102383:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626762752/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827670.102414:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827670/real 1787827670]  req@ffff8cd0c8eac000 x1874671626762368/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827695 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827670.102428:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626762368/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827695 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827670.102436:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827670.102439:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827670.102442:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626762368:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827670.102460:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626762496:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827670.102474:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827670.102482:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827670/real 1787827670]  req@ffff8cd0c8eacf00 x1874671626762496/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827695 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827670.102498:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626762496/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827695 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827670.102504:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827670.102506:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827670.102508:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626762496:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827670.102512:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626762624:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827670.102525:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827670.102531:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827670/real 1787827670]  req@ffff8cd0c8eae940 x1874671626762624/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827695 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827670.102539:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626762624/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827695 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827670.102544:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827670.102546:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827670.102548:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626762624:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827670.102551:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626762752:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827670.102561:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827670.102567:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827670/real 1787827670]  req@ffff8cd0c8eae1c0 x1874671626762752/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827695 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827670.102573:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626762752/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827695 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827670.102587:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827670.102589:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827670.102591:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626762752:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827670.294164:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827670.678253:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827671.318167:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827671.702276:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827672.342206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827672.726332:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827673.366222:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827673.750212:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827674.390161:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827674.774230:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827675.222180:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827675.222189:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827675.222194:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827675.222206:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1561
00000100:00080000:3.0:1787827675.222218:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827675.222227:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827675.222281:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626762880/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827675.222308:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827675.222311:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827675.222313:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827675.222316:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1561
00000100:00080000:3.0:1787827675.222319:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827675.222323:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827675.222332:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827675.222342:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626763008/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827675.222356:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827675.222373:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827675.222376:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827675.222380:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1561
00000100:00080000:3.0:1787827675.222385:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827675.222389:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827675.222394:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827675.222405:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626762880:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827675.222407:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626763136/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827675.222423:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827675.222425:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827675.222427:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827675.222431:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1561
00000100:00080000:3.0:1787827675.222434:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827675.222437:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827675.222441:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827675.222450:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626763264/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787827675.222478:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827675.222517:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827675/real 1787827675]  req@ffff8cd0c8eaf840 x1874671626762880/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827700 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827675.222533:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626762880/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827700 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827675.222543:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827675.222546:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827675.222550:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626762880:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827675.222556:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626763008:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827675.222573:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827675.222582:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827675/real 1787827675]  req@ffff8cd0c8eac780 x1874671626763008/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827700 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827675.222591:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626763008/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827700 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827675.222598:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827675.222600:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827675.222602:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626763008:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827675.222635:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626763136:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827675.222654:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827675.222663:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827675/real 1787827675]  req@ffff8cd0c8ead2c0 x1874671626763136/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827700 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827675.222672:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626763136/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827700 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827675.222679:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827675.222681:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827675.222683:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626763136:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827675.222688:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626763264:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827675.222700:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827675.222707:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827675/real 1787827675]  req@ffff8cd0c8eac3c0 x1874671626763264/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827700 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827675.222716:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626763264/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827700 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827675.222722:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827675.222724:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827675.222726:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626763264:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827675.414175:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827675.798251:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827676.438146:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827676.822318:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827677.462165:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827677.846228:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827678.486182:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827678.870301:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827679.510174:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827679.894341:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827680.342193:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827680.342202:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827680.342205:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827680.342215:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1566
00000100:00080000:3.0:1787827680.342224:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827680.342230:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827680.342264:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626763392/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827680.342301:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827680.342303:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827680.342305:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827680.342308:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1566
00000100:00080000:3.0:1787827680.342311:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827680.342314:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827680.342321:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827680.342331:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626763520/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827680.342338:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827680.342340:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827680.342342:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827680.342344:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1566
00000100:00080000:3.0:1787827680.342347:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827680.342350:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827680.342353:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827680.342363:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626763648/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827680.342371:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827680.342374:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827680.342375:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827680.342377:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1566
00000100:00080000:3.0:1787827680.342384:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827680.342387:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827680.342391:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827680.342404:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626763776/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827680.342420:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626763392:192.168.203.145@tcp:250:kworker.0
00000800:00000100:2.0:1787827680.342550:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827680.342631:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827680/real 1787827680]  req@ffff8cd0c8eada40 x1874671626763392/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827705 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827680.342691:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626763392/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827705 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827680.342738:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827680.342749:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827680.342758:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626763392:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827680.342776:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626763520:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827680.342827:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827680.342854:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827680/real 1787827680]  req@ffff8cd0c8eaf0c0 x1874671626763520/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827705 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827680.342880:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626763520/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827705 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827680.342898:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827680.342905:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827680.342912:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626763520:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827680.342926:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626763648:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827680.342966:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827680.342989:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827680/real 1787827680]  req@ffff8cd0c8eac3c0 x1874671626763648/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827705 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827680.343015:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626763648/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827705 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827680.343057:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827680.343065:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827680.343073:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626763648:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827680.343203:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626763776:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827680.343254:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827680.343278:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827680/real 1787827680]  req@ffff8cd0c8ead2c0 x1874671626763776/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827705 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827680.343304:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626763776/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827705 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827680.343321:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827680.343328:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827680.343335:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626763776:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827680.534187:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827680.918230:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827681.558143:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827681.942335:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827682.582187:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827682.966238:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827683.606150:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827683.990371:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827684.630169:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827685.014206:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827685.462140:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827685.462151:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827685.462155:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827685.462165:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1571
00000100:00080000:3.0:1787827685.462176:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827685.462183:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827685.462221:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626763904/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827685.462245:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827685.462248:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827685.462249:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827685.462252:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1571
00000100:00080000:3.0:1787827685.462255:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827685.462258:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827685.462267:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827685.462277:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626764032/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827685.462288:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626763904:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827685.462288:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827685.462313:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827685.462315:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827685.462319:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1571
00000100:00080000:3.0:1787827685.462323:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827685.462326:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827685.462330:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827685.462340:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626764160/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827685.462348:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827685.462351:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827685.462352:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827685.462355:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1571
00000100:00080000:3.0:1787827685.462358:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1787827685.462362:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787827685.462365:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827685.462370:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827685.462379:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626764288/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827685.462401:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827685/real 1787827685]  req@ffff8cd0c8eac780 x1874671626763904/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827710 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827685.462417:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626763904/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827710 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827685.462428:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827685.462432:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827685.462436:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626763904:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827685.462471:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626764032:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827685.462493:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827685.462514:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827685/real 1787827685]  req@ffff8cd0c8eaf840 x1874671626764032/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827710 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827685.462522:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626764032/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827710 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827685.462528:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827685.462530:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827685.462533:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626764032:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827685.462536:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626764160:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827685.462548:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827685.462554:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827685/real 1787827685]  req@ffff8cd0c8eae1c0 x1874671626764160/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827710 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827685.462561:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626764160/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827710 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827685.462566:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827685.462568:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827685.462569:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626764160:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827685.462582:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626764288:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827685.462592:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827685.462598:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827685/real 1787827685]  req@ffff8cd0c8eae940 x1874671626764288/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827710 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827685.462612:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626764288/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827710 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827685.462617:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827685.462619:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827685.462621:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626764288:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827685.654162:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827686.038300:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827686.678196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827687.062245:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827687.702216:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827688.086277:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827688.726149:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827689.110201:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827689.750162:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827690.134260:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827690.582151:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827690.582159:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827690.582163:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827690.582171:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1576
00000100:00080000:3.0:1787827690.582181:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827690.582187:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827690.582218:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626764416/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827690.582252:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827690.582254:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827690.582256:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827690.582259:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1576
00000100:00080000:3.0:1787827690.582262:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827690.582264:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827690.582278:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827690.582288:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626764544/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827690.582296:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827690.582299:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827690.582301:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827690.582303:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1576
00000100:00080000:3.0:1787827690.582306:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827690.582308:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827690.582311:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827690.582318:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626764672/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827690.582325:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827690.582328:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827690.582329:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827690.582331:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1576
00000100:00080000:3.0:1787827690.582343:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1787827690.582344:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626764416:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827690.582351:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827690.582358:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827690.582366:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626764800/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787827690.582470:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827690.582549:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827690/real 1787827690]  req@ffff8cd0c8eacf00 x1874671626764416/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827715 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827690.582570:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626764416/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827715 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827690.582583:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827690.582588:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827690.582593:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626764416:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827690.582602:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626764544:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827690.582629:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827690.582658:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827690/real 1787827690]  req@ffff8cd0c8eac000 x1874671626764544/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827715 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827690.582673:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626764544/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827715 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827690.582701:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827690.582705:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827690.582708:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626764544:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827690.582738:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626764672:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827690.582766:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827690.582778:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827690/real 1787827690]  req@ffff8cd0c8eae940 x1874671626764672/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827715 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827690.582791:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626764672/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827715 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827690.582800:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827690.582803:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827690.582807:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626764672:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827690.582814:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626764800:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827690.582835:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827690.582847:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827690/real 1787827690]  req@ffff8cd0c8eae1c0 x1874671626764800/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827715 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827690.582859:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626764800/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827715 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827690.582867:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827690.582871:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827690.582881:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626764800:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827690.774175:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827691.158198:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827691.798194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827692.182291:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827692.822239:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827693.206187:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827693.846236:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827694.230250:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827694.870221:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827695.254198:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827695.702160:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827695.702169:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827695.702173:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827695.702182:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1581
00000100:00080000:3.0:1787827695.702193:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827695.702199:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827695.702234:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626764928/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827695.702260:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827695.702263:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827695.702265:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827695.702268:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1581
00000100:00080000:3.0:1787827695.702271:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827695.702274:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827695.702282:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827695.702294:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626765056/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827695.702303:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626764928:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827695.702307:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827695.702325:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827695.702327:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827695.702332:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1581
00000100:00080000:3.0:1787827695.702337:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827695.702341:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827695.702346:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827695.702358:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626765184/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827695.702367:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827695.702387:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827695.702389:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827695.702394:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1581
00000800:00000100:2.0:1787827695.702396:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787827695.702398:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827695.702402:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827695.702407:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827695.702416:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626765312/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827695.702456:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827695/real 1787827695]  req@ffff8cd0c8eaf840 x1874671626764928/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827720 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827695.702475:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626764928/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827720 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827695.702484:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827695.702488:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827695.702491:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626764928:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827695.702513:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626765056:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827695.702535:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827695.702545:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827695/real 1787827695]  req@ffff8cd0c8eac780 x1874671626765056/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827720 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827695.702555:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626765056/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827720 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827695.702561:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827695.702564:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827695.702566:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626765056:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827695.702571:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626765184:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827695.702644:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827695.702667:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827695/real 1787827695]  req@ffff8cd0c8ead2c0 x1874671626765184/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827720 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827695.702678:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626765184/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827720 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827695.702684:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827695.702686:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827695.702688:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626765184:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827695.702692:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626765312:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827695.702716:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827695.702724:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827695/real 1787827695]  req@ffff8cd0c8eac3c0 x1874671626765312/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827720 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827695.702732:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626765312/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827720 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827695.702738:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827695.702740:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827695.702742:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626765312:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827695.894185:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827696.278231:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827696.918200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827697.302197:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827697.942166:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827698.326221:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827698.966159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827699.350201:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827699.990191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827700.374251:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827700.822198:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827700.822210:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827700.822215:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827700.822227:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1586
00000100:00080000:3.0:1787827700.822240:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827700.822264:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827700.822299:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626765440/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827700.822373:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827700.822377:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827700.822379:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827700.822384:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1586
00000100:00080000:3.0:1787827700.822389:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827700.822394:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827700.822404:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827700.822419:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626765568/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827700.822432:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827700.822436:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827700.822438:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827700.822442:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1586
00000100:00080000:3.0:1787827700.822445:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827700.822448:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827700.822453:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827700.822465:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626765696/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827700.822473:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626765440:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827700.822488:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827700.822498:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827700.822501:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827700.822506:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1586
00000100:00080000:3.0:1787827700.822518:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827700.822524:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827700.822530:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827700.822546:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626765824/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787827700.822600:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827700.822671:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827700/real 1787827700]  req@ffff8cd0c8eaf0c0 x1874671626765440/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827700.822687:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626765440/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827700.822704:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827700.822708:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827700.822711:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626765440:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827700.822718:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626765568:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827700.822739:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827700.822749:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827700/real 1787827700]  req@ffff8cd0c8eada40 x1874671626765568/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827700.822759:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626765568/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827700.822764:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827700.822767:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827700.822769:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626765568:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827700.822790:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626765696:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827700.822813:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827700.822820:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827700/real 1787827700]  req@ffff8cd0c8eac3c0 x1874671626765696/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827700.822829:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626765696/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827700.822835:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827700.822837:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827700.822839:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626765696:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827700.822849:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626765824:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827700.822863:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827700.822871:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827700/real 1787827700]  req@ffff8cd0c8ead2c0 x1874671626765824/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827700.822880:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626765824/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827700.822885:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827700.822887:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827700.822889:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626765824:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827701.014176:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827701.398246:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827702.038205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827702.422335:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827703.062152:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827703.446283:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827704.086203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827704.470234:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827705.110189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827705.494276:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827705.942176:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827705.942183:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827705.942187:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827705.942195:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1591
00000100:00080000:3.0:1787827705.942205:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827705.942213:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827705.942261:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626765952/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827705.942283:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827705.942286:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827705.942287:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827705.942290:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1591
00000100:00080000:3.0:1787827705.942294:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827705.942296:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827705.942316:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827705.942332:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626766080/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827705.942341:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1787827705.942345:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626765952:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827705.942358:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827705.942359:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827705.942363:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1591
00000100:00080000:3.0:1787827705.942367:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827705.942370:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827705.942379:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827705.942389:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626766208/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827705.942397:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827705.942400:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827705.942402:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827705.942404:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1591
00000100:00080000:3.0:1787827705.942406:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827705.942409:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827705.942412:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787827705.942416:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827705.942423:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626766336/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827705.942475:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827705/real 1787827705]  req@ffff8cd0c8eac780 x1874671626765952/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827705.942495:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626765952/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827705.942509:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827705.942513:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827705.942518:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626765952:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827705.942562:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626766080:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827705.942588:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827705.942602:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827705/real 1787827705]  req@ffff8cd0c8eaf840 x1874671626766080/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827705.942627:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626766080/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827705.942637:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827705.942640:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827705.942644:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626766080:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827705.942651:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626766208:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827705.942673:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827705.942685:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827705/real 1787827705]  req@ffff8cd0c8eae1c0 x1874671626766208/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827705.942698:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626766208/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827705.942706:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827705.942710:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827705.942713:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626766208:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827705.942720:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626766336:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827705.942738:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827705.942749:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827705/real 1787827705]  req@ffff8cd0c8eae940 x1874671626766336/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827705.942761:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626766336/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827705.942783:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827705.942787:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827705.942791:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626766336:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827706.134160:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827706.518370:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827707.158162:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827707.542282:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827708.182166:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827708.566198:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827709.206152:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827709.590221:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827710.230149:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827710.614217:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827711.062208:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827711.062217:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827711.062221:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827711.062229:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1596
00000100:00080000:2.0:1787827711.062239:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827711.062245:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827711.062278:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daad00 x1874671626766464/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827711.062303:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827711.062306:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827711.062307:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827711.062310:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1596
00000100:00080000:2.0:1787827711.062312:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827711.062315:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827711.062323:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827711.062335:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626766592/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827711.062339:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626766464:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787827711.062350:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827711.062353:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827711.062368:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827711.062372:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1596
00000100:00080000:2.0:1787827711.062376:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827711.062379:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827711.062384:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827711.062394:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626766720/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827711.062401:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827711.062405:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000800:00000100:3.0:1787827711.062406:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787827711.062408:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827711.062414:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1596
00000100:00080000:2.0:1787827711.062417:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827711.062421:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827711.062424:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827711.062433:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626766848/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827711.062449:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827711/real 1787827711]  req@ffff8cd0c3daad00 x1874671626766464/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827736 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827711.062465:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daad00 x1874671626766464/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827736 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827711.062493:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827711.062509:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827711.062513:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626766464:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827711.062534:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626766592:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827711.062563:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827711.062573:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827711/real 1787827711]  req@ffff8cd0c3daa580 x1874671626766592/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827736 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827711.062583:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626766592/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827736 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827711.062589:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827711.062592:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827711.062594:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626766592:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827711.062598:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626766720:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827711.062612:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827711.062619:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827711/real 1787827711]  req@ffff8cd0c3da92c0 x1874671626766720/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827736 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827711.062627:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626766720/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827736 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827711.062633:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827711.062635:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827711.062637:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626766720:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827711.062647:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626766848:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827711.062661:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827711.062668:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827711/real 1787827711]  req@ffff8cd0c3da9e00 x1874671626766848/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827736 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827711.062677:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626766848/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827736 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827711.062682:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827711.062684:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827711.062686:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626766848:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827711.254160:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827711.638204:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827712.278160:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827712.662240:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827713.302155:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827713.686220:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827714.326237:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827714.710244:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827715.350174:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827715.734203:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827716.182145:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827716.182154:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827716.182158:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827716.182167:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1602
00000100:00080000:2.0:1787827716.182176:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827716.182182:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827716.182213:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626766976/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827716.182246:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827716.182249:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827716.182250:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827716.182253:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1602
00000100:00080000:2.0:1787827716.182256:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827716.182259:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827716.182267:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827716.182278:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626767104/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827716.182288:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787827716.182288:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626766976:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787827716.182295:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827716.182299:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827716.182303:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1602
00000100:00080000:2.0:1787827716.182305:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827716.182307:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827716.182310:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827716.182320:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626767232/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827716.182327:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827716.182329:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827716.182331:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827716.182334:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1602
00000100:00080000:2.0:1787827716.182342:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827716.182346:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827716.182349:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827716.182359:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626767360/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827716.182397:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626767104:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827716.182427:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626767232:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827716.182448:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626767360:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827716.374191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827716.758226:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827717.398158:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827717.782217:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827718.422186:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827718.806278:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:1.0:1787827719.254323:0:2344:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:1.0:1787827719.254364:0:2344:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1787827719.254378:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827719.254415:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827719.254422:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827719.254427:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:3.0:1787827719.254528:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827716/real 1787827719]  req@ffff8cd0c3daa1c0 x1874671626767104/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827741 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827719.254549:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626767104/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827741 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827719.254563:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827719.254569:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827719.254573:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626767104:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827719.254598:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827716/real 1787827719]  req@ffff8cd0c3dab840 x1874671626767232/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827741 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827719.254613:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626767232/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827741 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827719.254622:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827719.254625:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827719.254628:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626767232:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827719.254634:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827716/real 1787827719]  req@ffff8cd0c3da8000 x1874671626767360/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827741 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827719.254646:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626767360/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827741 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827719.254654:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827719.254658:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827719.254661:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626767360:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827719.254667:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827716/real 1787827719]  req@ffff8cd0c3dabc00 x1874671626766976/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827741 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827719.254679:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626766976/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827741 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827719.254686:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827719.254689:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827719.254697:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626766976:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787827719.446213:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827719.830193:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827720.470207:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787827720.854260:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787827721.302206:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827721.302215:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787827721.302218:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827721.302226:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1607
00000100:00080000:0.0:1787827721.302236:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827721.302243:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787827721.302277:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626767488/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827721.302318:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827721.302321:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827721.302322:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827721.302325:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1607
00000100:00080000:0.0:1787827721.302328:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827721.302331:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827721.302339:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827721.302349:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626767616/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827721.302357:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827721.302360:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787827721.302361:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1787827721.302366:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626767488:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787827721.302367:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1607
00000100:00080000:0.0:1787827721.302373:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827721.302377:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827721.302381:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827721.302388:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626767744/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827721.302395:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827721.302397:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827721.302399:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827721.302401:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1607
00000100:00080000:0.0:1787827721.302410:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827721.302413:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827721.302417:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827721.302439:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626767872/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787827721.302446:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827721.302509:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827721/real 1787827721]  req@ffff8cd0c8e9cf00 x1874671626767488/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827746 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827721.302529:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626767488/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827746 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827721.302540:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827721.302545:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827721.302549:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626767488:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827721.302573:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626767616:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827721.302598:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827721.302612:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827721/real 1787827721]  req@ffff8cd0c8e9c3c0 x1874671626767616/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827746 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827721.302625:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626767616/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827746 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827721.302634:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827721.302638:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827721.302640:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626767616:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827721.302647:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626767744:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827721.302668:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827721.302679:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827721/real 1787827721]  req@ffff8cd0c8e9ed00 x1874671626767744/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827746 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827721.302702:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626767744/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827746 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827721.302710:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827721.302713:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827721.302716:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626767744:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827721.302722:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626767872:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827721.302745:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827721.302756:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827721/real 1787827721]  req@ffff8cd0c8e9f840 x1874671626767872/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827746 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827721.302767:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626767872/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827746 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827721.302775:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827721.302778:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827721.302781:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626767872:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827721.494231:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827721.878222:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827722.518192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827722.902328:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827723.542160:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827723.926215:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827724.566152:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827724.950246:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827725.590161:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827725.974212:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827726.422186:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827726.422197:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827726.422202:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827726.422221:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1612
00000100:00080000:3.0:1787827726.422234:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827726.422239:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827726.422274:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626768000/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827726.422312:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827726.422315:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827726.422316:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827726.422319:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1612
00000100:00080000:3.0:1787827726.422323:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827726.422326:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827726.422335:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827726.422346:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626768128/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827726.422351:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626768000:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827726.422358:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827726.422361:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827726.422363:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827726.422366:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1612
00000100:00080000:3.0:1787827726.422368:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827726.422371:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827726.422375:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827726.422384:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626768256/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827726.422406:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827726.422409:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827726.422411:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827726.422414:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1612
00000100:00080000:3.0:1787827726.422424:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1787827726.422427:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787827726.422431:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827726.422437:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827726.422447:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626768384/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827726.422470:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827726/real 1787827726]  req@ffff8cd0c8eac000 x1874671626768000/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827751 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827726.422484:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626768000/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827751 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827726.422492:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827726.422495:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827726.422498:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626768000:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827726.422517:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626768128:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827726.422533:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827726.422542:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827726/real 1787827726]  req@ffff8cd0c8eacf00 x1874671626768128/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827751 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827726.422550:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626768128/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827751 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827726.422555:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827726.422558:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827726.422566:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626768128:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827726.422570:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626768256:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827726.422586:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827726.422593:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827726/real 1787827726]  req@ffff8cd0c8eae940 x1874671626768256/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827751 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827726.422601:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626768256/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827751 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827726.422606:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827726.422608:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827726.422610:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626768256:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827726.422613:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626768384:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827726.422624:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827726.422630:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827726/real 1787827726]  req@ffff8cd0c8eae1c0 x1874671626768384/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827751 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827726.422639:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626768384/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827751 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827726.422647:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827726.422650:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827726.422654:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626768384:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827726.614161:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827726.998206:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827727.638238:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827728.022226:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827728.662213:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827729.046205:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827729.686226:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827730.070172:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827730.710174:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827731.094236:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787827731.542255:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827731.542264:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787827731.542268:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827731.542278:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1617
00000100:00080000:0.0:1787827731.542305:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827731.542312:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787827731.542352:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626768512/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827731.542433:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827731.542435:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827731.542437:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827731.542454:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1617
00000100:00080000:0.0:1787827731.542458:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827731.542462:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827731.542470:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827731.542475:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626768512:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827731.542484:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626768640/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827731.542493:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827731.542496:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787827731.542497:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827731.542499:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1617
00000100:00080000:0.0:1787827731.542502:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827731.542504:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827731.542507:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827731.542517:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626768768/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827731.542527:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827731.542530:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827731.542532:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1787827731.542536:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787827731.542540:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1617
00000100:00080000:0.0:1787827731.542547:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827731.542553:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827731.542559:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827731.542572:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626768896/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827731.542583:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827731/real 1787827731]  req@ffff8cd0c8e9d680 x1874671626768512/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827756 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827731.542630:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626768512/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827756 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827731.542652:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827731.542656:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827731.542659:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626768512:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827731.542681:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626768640:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827731.542732:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827731.542746:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827731/real 1787827731]  req@ffff8cd0c8e9f480 x1874671626768640/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827756 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827731.542768:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626768640/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827756 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827731.542777:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827731.542780:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827731.542783:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626768640:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827731.542789:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626768768:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827731.542808:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827731.542817:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827731/real 1787827731]  req@ffff8cd0c8e9cb40 x1874671626768768/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827756 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827731.542828:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626768768/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827756 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827731.542852:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827731.542856:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827731.542859:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626768768:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827731.542865:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626768896:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827731.542886:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827731.542896:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827731/real 1787827731]  req@ffff8cd0c8e9c000 x1874671626768896/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827756 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827731.542906:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626768896/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827756 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827731.542914:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827731.542916:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827731.542919:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626768896:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827731.734230:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827732.118166:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827732.758234:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827733.142215:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827733.782220:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827734.166204:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827734.806205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827735.190212:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827735.830249:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827736.214211:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827736.662228:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827736.662239:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827736.662244:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827736.662254:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1622
00000100:00080000:3.0:1787827736.662267:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827736.662274:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827736.662334:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626769024/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827736.662384:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827736.662387:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827736.662389:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827736.662392:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1622
00000100:00080000:3.0:1787827736.662396:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827736.662400:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827736.662412:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827736.662424:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626769152/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827736.662435:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626769024:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827736.662441:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827736.662471:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827736.662474:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827736.662498:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1622
00000100:00080000:3.0:1787827736.662507:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827736.662515:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:2.0:1787827736.662523:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:3.0:1787827736.662531:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827736.662574:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626769280/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827736.662579:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827736/real 1787827736]  req@ffff8cd0c8eaf840 x1874671626769024/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827761 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827736.662614:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827736.662619:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827736.662621:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1787827736.662636:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626769024/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827761 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827736.662657:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1622
00000100:00080000:3.0:1787827736.662666:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827736.662667:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827736.662679:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00080000:3.0:1787827736.662679:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827736.662704:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626769024:192.168.203.145@tcp:250:kworker.0
00000008:00100000:3.0:1787827736.662710:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827736.662724:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626769408/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827736.662755:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626769152:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827736.662788:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827736.662815:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827736/real 1787827736]  req@ffff8cd0c8eac780 x1874671626769152/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827761 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827736.662828:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626769152/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827761 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827736.662846:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827736.662850:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827736.662853:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626769152:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827736.662872:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626769280:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827736.662896:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827736.662908:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827736/real 1787827736]  req@ffff8cd0c8ead2c0 x1874671626769280/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827761 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827736.662916:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626769280/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827761 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827736.662922:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827736.662924:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827736.662927:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626769280:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827736.662931:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626769408:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827736.662945:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827736.662952:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827736/real 1787827736]  req@ffff8cd0c8eac3c0 x1874671626769408/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827761 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827736.662965:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626769408/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827761 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827736.662971:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827736.662973:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827736.662975:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626769408:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827736.854242:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827737.238334:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827737.878195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827738.262391:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827738.902155:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827739.286213:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827739.926167:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827740.310278:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827740.950193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827741.334246:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787827741.782314:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827741.782325:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787827741.782329:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827741.782341:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1627
00000100:00080000:0.0:1787827741.782353:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827741.782361:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787827741.782410:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d2c0 x1874671626769536/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827741.782441:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827741.782444:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827741.782446:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827741.782448:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1627
00000100:00080000:0.0:1787827741.782451:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827741.782453:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827741.782464:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827741.782498:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626769664/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827741.782508:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827741.782515:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:2.0:1787827741.782517:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626769536:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787827741.782518:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827741.782530:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1627
00000100:00080000:0.0:1787827741.782536:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827741.782539:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827741.782544:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827741.782554:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626769792/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827741.782561:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827741.782564:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827741.782566:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827741.782569:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1627
00000100:00080000:0.0:1787827741.782572:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827741.782575:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827741.782579:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827741.782589:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626769920/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787827741.782626:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827741.782672:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827741/real 1787827741]  req@ffff8cd0c8e9d2c0 x1874671626769536/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827741.782690:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d2c0 x1874671626769536/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827741.782701:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827741.782705:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827741.782709:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626769536:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827741.782734:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626769664:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827741.782755:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827741.782766:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827741/real 1787827741]  req@ffff8cd0c8e9c000 x1874671626769664/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827741.782776:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626769664/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827741.782819:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827741.782823:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827741.782826:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626769664:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827741.782832:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626769792:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827741.782854:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827741.782864:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827741/real 1787827741]  req@ffff8cd0c8e9cb40 x1874671626769792/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827741.782874:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626769792/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827741.782881:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827741.782883:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827741.782886:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626769792:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827741.782891:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626769920:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827741.782905:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827741.782913:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827741/real 1787827741]  req@ffff8cd0c8e9f480 x1874671626769920/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827741.782923:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626769920/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827741.782929:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827741.782931:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827741.782940:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626769920:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827741.974298:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827742.358196:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827742.998178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827743.382221:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827744.022243:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827744.406230:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827745.046273:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827745.430299:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827746.070167:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827746.454232:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827746.902197:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827746.902205:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827746.902209:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827746.902218:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1632
00000100:00080000:3.0:1787827746.902228:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827746.902234:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827746.902280:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626770048/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827746.902318:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827746.902320:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827746.902322:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827746.902325:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1632
00000100:00080000:3.0:1787827746.902328:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827746.902331:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827746.902338:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827746.902348:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626770176/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827746.902356:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827746.902359:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827746.902360:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827746.902362:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1632
00000100:00080000:3.0:1787827746.902364:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827746.902366:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827746.902370:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827746.902375:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626770048:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827746.902380:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626770304/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827746.902389:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827746.902391:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827746.902393:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827746.902395:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1632
00000100:00080000:3.0:1787827746.902402:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827746.902405:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827746.902410:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827746.902419:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626770432/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787827746.902440:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827746.902475:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827746/real 1787827746]  req@ffff8cd0c8eada40 x1874671626770048/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827746.902495:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626770048/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827746.902506:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827746.902511:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827746.902515:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626770048:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827746.902524:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626770176:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827746.902551:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827746.902566:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827746/real 1787827746]  req@ffff8cd0c8eaf0c0 x1874671626770176/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827746.902579:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626770176/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827746.902588:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827746.902592:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827746.902594:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626770176:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827746.902615:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626770304:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827746.902632:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827746.902648:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827746/real 1787827746]  req@ffff8cd0c8eac3c0 x1874671626770304/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827746.902656:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626770304/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827746.902662:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827746.902664:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827746.902666:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626770304:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827746.902670:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626770432:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827746.902682:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827746.902697:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827746/real 1787827746]  req@ffff8cd0c8ead2c0 x1874671626770432/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827746.902705:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626770432/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827746.902710:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827746.902712:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827746.902714:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626770432:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827747.094193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827747.478236:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827748.118193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827748.502292:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827749.142274:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827749.526226:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827750.166175:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827750.550215:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827751.190234:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827751.574186:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787827752.022176:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827752.022185:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787827752.022189:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827752.022196:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1637
00000100:00080000:0.0:1787827752.022206:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827752.022225:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787827752.022258:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626770560/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827752.022283:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827752.022286:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827752.022287:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827752.022290:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1637
00000100:00080000:0.0:1787827752.022293:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827752.022295:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827752.022303:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827752.022315:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626770688/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827752.022323:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827752.022326:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787827752.022339:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827752.022344:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1637
00000100:00080000:0.0:1787827752.022351:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1787827752.022351:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626770560:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787827752.022364:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827752.022371:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827752.022385:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626770816/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827752.022395:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827752.022398:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827752.022400:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827752.022403:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1637
00000100:00080000:0.0:1787827752.022405:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827752.022408:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827752.022410:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827752.022420:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626770944/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787827752.022453:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827752.022506:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827752/real 1787827752]  req@ffff8cd0c8e9d680 x1874671626770560/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827777 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827752.022528:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626770560/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827777 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827752.022558:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827752.022562:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827752.022566:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626770560:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827752.022573:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626770688:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827752.022596:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827752.022606:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827752/real 1787827752]  req@ffff8cd0c8e9f840 x1874671626770688/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827777 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827752.022615:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626770688/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827777 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827752.022621:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827752.022623:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827752.022626:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626770688:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827752.022652:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626770816:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827752.022666:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827752.022672:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827752/real 1787827752]  req@ffff8cd0c8e9ed00 x1874671626770816/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827777 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827752.022680:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626770816/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827777 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827752.022686:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827752.022688:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827752.022690:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626770816:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827752.022699:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626770944:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827752.022713:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827752.022720:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827752/real 1787827752]  req@ffff8cd0c8e9c3c0 x1874671626770944/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827777 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827752.022728:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626770944/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827777 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827752.022733:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827752.022735:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827752.022737:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626770944:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827752.214168:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827752.598302:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827753.238189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827753.622175:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827754.262192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827754.646251:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827755.286179:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827755.670165:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827756.310240:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827756.694197:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827757.142215:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827757.142224:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827757.142227:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827757.142237:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1642
00000100:00080000:3.0:1787827757.142246:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827757.142253:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827757.142288:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626771072/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827757.142312:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827757.142315:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827757.142316:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827757.142319:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1642
00000100:00080000:3.0:1787827757.142321:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827757.142324:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827757.142332:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827757.142342:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626771200/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827757.142352:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626771072:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827757.142352:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827757.142368:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827757.142370:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827757.142374:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1642
00000100:00080000:3.0:1787827757.142377:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827757.142381:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827757.142386:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827757.142396:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626771328/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827757.142404:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827757.142406:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827757.142408:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1787827757.142412:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787827757.142413:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1642
00000100:00080000:3.0:1787827757.142420:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827757.142423:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827757.142427:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827757.142440:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626771456/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827757.142451:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827757/real 1787827757]  req@ffff8cd0c8eac780 x1874671626771072/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827782 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827757.142492:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626771072/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827782 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827757.142502:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827757.142505:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827757.142508:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626771072:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827757.142527:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626771200:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827757.142544:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827757.142553:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827757/real 1787827757]  req@ffff8cd0c8eaf840 x1874671626771200/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827782 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827757.142574:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626771200/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827782 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827757.142584:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827757.142588:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827757.142592:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626771200:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827757.142598:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626771328:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827757.142622:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827757.142633:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827757/real 1787827757]  req@ffff8cd0c8eae1c0 x1874671626771328/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827782 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827757.142641:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626771328/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827782 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827757.142647:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827757.142649:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827757.142651:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626771328:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827757.142655:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626771456:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827757.142667:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827757.142674:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827757/real 1787827757]  req@ffff8cd0c8eae940 x1874671626771456/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827782 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827757.142686:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626771456/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827782 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827757.142707:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827757.142711:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827757.142715:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626771456:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827757.334173:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827757.718160:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827758.358149:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827758.742235:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827759.382173:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827759.766249:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827760.406204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827760.790235:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827761.430228:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827761.814231:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827762.262243:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827762.262254:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827762.262258:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827762.262270:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1648
00000100:00080000:3.0:1787827762.262282:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827762.262289:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827762.262334:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626771584/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827762.262380:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827762.262383:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827762.262384:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827762.262387:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1648
00000100:00080000:3.0:1787827762.262390:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827762.262393:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827762.262402:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827762.262412:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626771712/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827762.262421:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827762.262424:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827762.262426:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827762.262428:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1648
00000100:00080000:3.0:1787827762.262430:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827762.262432:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827762.262436:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827762.262440:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626771584:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827762.262448:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626771840/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827762.262456:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827762.262458:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827762.262459:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827762.262462:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1648
00000100:00080000:3.0:1787827762.262470:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827762.262473:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827762.262477:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827762.262487:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626771968/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787827762.262519:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827762.262564:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827762/real 1787827762]  req@ffff8cd0c8eacf00 x1874671626771584/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827787 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827762.262578:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626771584/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827787 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827762.262588:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827762.262591:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827762.262594:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626771584:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827762.262599:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626771712:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827762.262615:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827762.262623:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827762/real 1787827762]  req@ffff8cd0c8eac000 x1874671626771712/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827787 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827762.262631:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626771712/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827787 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827762.262637:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827762.262639:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827762.262641:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626771712:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827762.262671:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626771840:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827762.262687:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827762.262695:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827762/real 1787827762]  req@ffff8cd0c8eae940 x1874671626771840/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827787 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827762.262702:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626771840/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827787 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827762.262708:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827762.262710:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827762.262712:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626771840:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827762.262716:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626771968:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827762.262727:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827762.262732:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827762/real 1787827762]  req@ffff8cd0c8eae1c0 x1874671626771968/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827787 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827762.262740:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626771968/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827787 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827762.262745:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827762.262747:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827762.262748:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626771968:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827762.454233:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827762.838338:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827763.478187:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827763.862231:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827764.502236:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827764.886220:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827765.526176:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827765.910228:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827766.550213:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827766.934215:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827767.382169:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827767.382178:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827767.382182:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827767.382190:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1653
00000100:00080000:3.0:1787827767.382199:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827767.382205:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827767.382236:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626772096/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827767.382260:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827767.382264:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827767.382266:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827767.382270:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1653
00000100:00080000:3.0:1787827767.382274:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827767.382279:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827767.382289:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827767.382304:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626772224/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827767.382309:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626772096:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827767.382321:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827767.382337:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827767.382339:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827767.382343:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1653
00000100:00080000:3.0:1787827767.382347:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827767.382350:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827767.382355:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827767.382366:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626772352/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827767.382373:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:2.0:1787827767.382377:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787827767.382378:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827767.382382:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827767.382386:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1653
00000100:00080000:3.0:1787827767.382389:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827767.382393:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827767.382397:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827767.382405:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626772480/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827767.382428:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827767/real 1787827767]  req@ffff8cd0c8eaf840 x1874671626772096/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827792 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827767.382460:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626772096/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827792 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827767.382471:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827767.382474:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827767.382477:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626772096:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827767.382498:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626772224:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827767.382521:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827767.382535:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827767/real 1787827767]  req@ffff8cd0c8eac780 x1874671626772224/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827792 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827767.382548:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626772224/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827792 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827767.382557:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827767.382560:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827767.382564:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626772224:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827767.382571:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626772352:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827767.382588:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827767.382597:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827767/real 1787827767]  req@ffff8cd0c8ead2c0 x1874671626772352/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827792 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827767.382604:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626772352/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827792 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827767.382628:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827767.382630:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827767.382633:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626772352:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827767.382637:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626772480:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827767.382653:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827767.382660:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827767/real 1787827767]  req@ffff8cd0c8eac3c0 x1874671626772480/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827792 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827767.382667:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626772480/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827792 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827767.382672:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827767.382674:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827767.382676:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626772480:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827767.574149:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827767.958254:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827768.598158:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827768.982230:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827769.622180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827770.006289:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827770.646158:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827771.030250:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827771.670169:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827772.054216:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827772.502178:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827772.502186:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827772.502189:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827772.502198:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1658
00000100:00080000:3.0:1787827772.502207:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827772.502212:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827772.502246:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626772608/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827772.502281:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827772.502284:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827772.502285:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827772.502288:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1658
00000100:00080000:3.0:1787827772.502291:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827772.502293:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827772.502301:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827772.502312:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626772736/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827772.502320:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1787827772.502324:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626772608:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827772.502325:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827772.502329:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827772.502335:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1658
00000100:00080000:3.0:1787827772.502339:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827772.502344:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827772.502350:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827772.502365:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626772864/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827772.502376:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827772.502379:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827772.502381:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1787827772.502385:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787827772.502389:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1658
00000100:00080000:3.0:1787827772.502399:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827772.502403:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827772.502407:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827772.502417:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626772992/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827772.502427:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827772/real 1787827772]  req@ffff8cd0c8eaf0c0 x1874671626772608/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827797 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827772.502442:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626772608/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827797 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827772.502450:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827772.502453:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827772.502456:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626772608:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827772.502482:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626772736:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827772.502500:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827772.502509:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827772/real 1787827772]  req@ffff8cd0c8eada40 x1874671626772736/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827797 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827772.502518:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626772736/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827797 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827772.502523:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827772.502526:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827772.502528:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626772736:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827772.502531:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626772864:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827772.502542:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827772.502548:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827772/real 1787827772]  req@ffff8cd0c8eac3c0 x1874671626772864/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827797 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827772.502555:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626772864/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827797 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827772.502560:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827772.502562:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827772.502564:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626772864:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827772.502568:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626772992:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827772.502577:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827772.502586:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827772/real 1787827772]  req@ffff8cd0c8ead2c0 x1874671626772992/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827797 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827772.502606:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626772992/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827797 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827772.502615:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827772.502618:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827772.502622:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626772992:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827772.694148:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827773.078173:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827773.718233:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827774.102230:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827774.742180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827775.126162:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827775.766203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827776.150248:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827776.790175:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827777.174187:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827777.622246:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827777.622261:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827777.622266:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827777.622278:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1663
00000100:00080000:3.0:1787827777.622302:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827777.622310:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827777.622355:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626773120/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827777.622394:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827777.622399:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827777.622401:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827777.622406:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1663
00000100:00080000:3.0:1787827777.622410:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827777.622415:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827777.622427:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827777.622441:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626773248/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827777.622451:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1787827777.622453:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626773120:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827777.622471:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827777.622473:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827777.622476:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1663
00000100:00080000:3.0:1787827777.622480:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827777.622484:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827777.622488:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827777.622506:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626773376/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827777.622514:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827777.622517:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827777.622519:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827777.622522:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1663
00000100:00080000:3.0:1787827777.622526:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827777.622529:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827777.622532:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827777.622540:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626773504/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787827777.622545:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827777.622594:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827777/real 1787827777]  req@ffff8cd0c8eac780 x1874671626773120/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827802 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827777.622610:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626773120/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827802 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827777.622620:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827777.622623:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827777.622627:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626773120:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827777.622649:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626773248:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827777.622670:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827777.622684:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827777/real 1787827777]  req@ffff8cd0c8eaf840 x1874671626773248/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827802 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827777.622698:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626773248/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827802 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827777.622721:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827777.622724:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827777.622727:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626773248:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827777.622731:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626773376:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827777.622750:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827777.622761:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827777/real 1787827777]  req@ffff8cd0c8eae1c0 x1874671626773376/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827802 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827777.622774:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626773376/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827802 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827777.622782:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827777.622786:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827777.622790:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626773376:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827777.622797:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626773504:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827777.622817:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827777.622827:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827777/real 1787827777]  req@ffff8cd0c8eae940 x1874671626773504/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827802 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827777.622840:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626773504/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827802 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827777.622848:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827777.622852:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827777.622862:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626773504:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787827777.814289:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827778.198242:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827778.838182:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827779.222260:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827779.862236:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827780.246224:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827780.886188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827781.270292:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:1.0:1787827781.910215:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827782.294222:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787827782.742197:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827782.742207:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787827782.742212:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827782.742222:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1668
00000100:00080000:0.0:1787827782.742236:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827782.742243:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787827782.742286:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626773632/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827782.742312:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827782.742336:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827782.742338:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827782.742341:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1668
00000100:00080000:0.0:1787827782.742345:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827782.742349:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827782.742360:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827782.742363:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626773632:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827782.742373:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626773760/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827782.742384:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827782.742387:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787827782.742389:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827782.742393:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1668
00000100:00080000:0.0:1787827782.742395:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827782.742399:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827782.742402:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827782.742412:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626773888/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827782.742420:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827782.742424:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827782.742425:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827782.742429:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1668
00000100:00080000:0.0:1787827782.742433:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827782.742436:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827782.742444:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827782.742454:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626774016/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827782.742477:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626773760:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827782.742506:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626773888:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827782.742532:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626774016:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827782.934195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827783.318254:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827783.958185:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827784.342199:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827784.982160:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827785.366299:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:1.0:1787827785.814478:0:2346:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:1.0:1787827785.814536:0:2346:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1787827785.814558:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827785.814605:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827785.814616:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827785.814625:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:2.0:1787827785.814666:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827782/real 1787827785]  req@ffff8cd0c8e9e940 x1874671626773760/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827807 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827785.814698:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626773760/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827807 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827785.814713:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827785.814719:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827785.814724:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626773760:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827785.814734:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827782/real 1787827785]  req@ffff8cd0c8e9e580 x1874671626773888/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827807 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827785.814748:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626773888/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827807 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827785.814758:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827785.814762:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827785.814807:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626773888:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827785.814817:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827782/real 1787827785]  req@ffff8cd0c8e9de00 x1874671626774016/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827807 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827785.814835:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626774016/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827807 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827785.814845:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827785.814849:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827785.814854:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626774016:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827785.814862:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827782/real 1787827785]  req@ffff8cd0c8e9cf00 x1874671626773632/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827807 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827785.814877:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626773632/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827807 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827785.814887:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827785.814892:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827785.814896:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626773632:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827786.006220:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827786.390218:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827787.030173:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827787.414244:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827787.862206:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827787.862214:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827787.862217:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827787.862227:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1673
00000100:00080000:3.0:1787827787.862236:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827787.862242:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827787.862277:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626774144/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827787.862311:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827787.862314:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827787.862316:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827787.862318:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1673
00000100:00080000:3.0:1787827787.862321:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827787.862324:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827787.862332:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827787.862351:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626774144:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827787.862352:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626774272/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827787.862365:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827787.862368:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827787.862369:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827787.862372:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1673
00000100:00080000:3.0:1787827787.862374:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827787.862377:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827787.862382:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827787.862396:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626774400/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827787.862407:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827787.862411:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827787.862413:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827787.862417:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1673
00000800:00000100:2.0:1787827787.862424:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787827787.862430:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827787.862438:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827787.862445:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827787.862458:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626774528/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827787.862472:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827787/real 1787827787]  req@ffff8cd0c8eac000 x1874671626774144/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827812 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827787.862493:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626774144/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827812 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827787.862516:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827787.862521:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827787.862526:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626774144:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827787.862550:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626774272:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827787.862586:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827787.862601:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827787/real 1787827787]  req@ffff8cd0c8eacf00 x1874671626774272/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827812 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827787.862614:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626774272/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827812 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827787.862623:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827787.862626:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827787.862630:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626774272:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827787.862636:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626774400:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827787.862658:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827787.862670:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827787/real 1787827787]  req@ffff8cd0c8eae940 x1874671626774400/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827812 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827787.862695:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626774400/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827812 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827787.862703:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827787.862707:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827787.862710:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626774400:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827787.862724:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626774528:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827787.862747:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827787.862760:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827787/real 1787827787]  req@ffff8cd0c8eae1c0 x1874671626774528/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827812 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827787.862773:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626774528/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827812 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827787.862782:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827787.862785:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827787.862789:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626774528:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827788.054155:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827788.438220:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827789.078153:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827789.462304:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827790.102235:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827790.486204:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827791.126218:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827791.510209:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827792.150202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827792.534191:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827792.982153:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827792.982162:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827792.982166:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827792.982174:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1678
00000100:00080000:3.0:1787827792.982195:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827792.982201:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827792.982250:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626774656/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827792.982271:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827792.982274:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827792.982275:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827792.982278:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1678
00000100:00080000:3.0:1787827792.982280:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827792.982283:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827792.982290:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827792.982299:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626774784/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827792.982306:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827792.982320:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827792.982322:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827792.982326:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1678
00000100:00080000:3.0:1787827792.982330:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827792.982333:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827792.982338:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827792.982347:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626774912/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827792.982356:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827792.982364:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827792.982366:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1787827792.982367:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626774656:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827792.982376:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1678
00000100:00080000:3.0:1787827792.982382:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827792.982386:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827792.982389:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827792.982399:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626775040/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787827792.982459:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827792.982578:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827792/real 1787827792]  req@ffff8cd0c8eaf840 x1874671626774656/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827817 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827792.982629:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626774656/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827817 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827792.982645:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827792.982652:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827792.982657:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626774656:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827792.982679:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626774784:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827792.982730:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827792.982742:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827792/real 1787827792]  req@ffff8cd0c8eac780 x1874671626774784/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827817 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827792.982751:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626774784/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827817 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827792.982758:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827792.982761:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827792.982779:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626774784:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827792.982825:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626774912:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827792.982872:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827792.982886:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827792/real 1787827792]  req@ffff8cd0c8ead2c0 x1874671626774912/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827817 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827792.982912:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626774912/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827817 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827792.982918:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827792.982921:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827792.982923:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626774912:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827792.982956:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626775040:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827792.982973:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827792.982982:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827792/real 1787827792]  req@ffff8cd0c8eac3c0 x1874671626775040/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827817 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827792.982997:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626775040/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827817 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827792.983013:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827792.983015:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827792.983017:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626775040:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827793.174198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827793.558233:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827794.198148:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827794.582199:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827795.222177:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827795.606177:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827796.246158:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827796.630211:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827797.270186:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827797.654187:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787827798.102193:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827798.102202:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787827798.102206:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827798.102215:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1683
00000100:00080000:0.0:1787827798.102224:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827798.102230:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787827798.102265:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626775168/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827798.102289:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827798.102292:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827798.102307:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827798.102310:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1683
00000100:00080000:0.0:1787827798.102314:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827798.102317:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827798.102326:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827798.102337:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626775296/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827798.102347:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827798.102350:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787827798.102352:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827798.102355:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1683
00000100:00080000:0.0:1787827798.102357:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827798.102360:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827798.102364:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827798.102371:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626775168:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787827798.102374:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626775424/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827798.102385:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827798.102388:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827798.102389:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827798.102392:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1683
00000100:00080000:0.0:1787827798.102395:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827798.102397:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827798.102401:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827798.102417:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626775552/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787827798.102441:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827798.102487:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827798/real 1787827798]  req@ffff8cd0c8e9fc00 x1874671626775168/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827823 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827798.102503:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626775168/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827823 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827798.102512:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827798.102516:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827798.102520:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626775168:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827798.102526:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626775296:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827798.102542:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827798.102551:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827798/real 1787827798]  req@ffff8cd0c8e9cf00 x1874671626775296/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827823 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827798.102570:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626775296/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827823 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827798.102576:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827798.102579:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827798.102581:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626775296:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827798.102602:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626775424:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827798.102618:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827798.102626:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827798/real 1787827798]  req@ffff8cd0c8e9de00 x1874671626775424/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827823 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827798.102645:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626775424/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827823 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827798.102652:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827798.102654:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827798.102656:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626775424:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827798.102661:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626775552:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827798.102673:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827798.102680:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827798/real 1787827798]  req@ffff8cd0c8e9e580 x1874671626775552/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827823 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827798.102688:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626775552/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827823 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827798.102704:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827798.102707:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827798.102709:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626775552:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827798.294196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827798.678188:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827799.318250:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827799.702198:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827800.342155:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827800.726202:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827801.366226:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827801.750260:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827802.390178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827802.774237:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827803.222247:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827803.222283:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827803.222289:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827803.222301:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1689
00000100:00080000:1.0:1787827803.222315:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827803.222324:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827803.222371:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626775680/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827803.222403:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827803.222406:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827803.222408:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827803.222412:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1689
00000100:00080000:1.0:1787827803.222415:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827803.222420:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827803.222431:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827803.222444:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626775808/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827803.222456:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626775680:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787827803.222457:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827803.222466:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827803.222468:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827803.222472:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1689
00000100:00080000:1.0:1787827803.222475:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827803.222479:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827803.222484:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827803.222497:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626775936/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827803.222516:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827803.222521:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827803.222523:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827803.222527:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1689
00000100:00080000:1.0:1787827803.222533:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1787827803.222534:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787827803.222542:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827803.222550:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827803.222574:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626776064/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827803.222596:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827803/real 1787827803]  req@ffff8cd0d48f7840 x1874671626775680/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827828 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827803.222613:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626775680/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827828 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827803.222622:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827803.222626:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827803.222629:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626775680:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827803.222652:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626775808:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827803.222678:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827803.222692:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827803/real 1787827803]  req@ffff8cd0d48f5a40 x1874671626775808/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827828 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827803.222707:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626775808/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827828 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827803.222720:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827803.222735:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827803.222737:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626775808:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827803.222742:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626775936:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827803.222761:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827803.222769:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827803/real 1787827803]  req@ffff8cd0d48f43c0 x1874671626775936/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827828 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827803.222777:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626775936/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827828 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827803.222783:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827803.222785:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827803.222787:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626775936:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827803.222791:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626776064:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827803.222807:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827803.222813:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827803/real 1787827803]  req@ffff8cd0d48f70c0 x1874671626776064/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827828 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827803.222820:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626776064/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827828 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827803.222825:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827803.222827:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827803.222834:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626776064:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827803.414163:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827803.798273:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827804.438230:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827804.822329:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827805.462226:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827805.846210:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827806.486197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827806.870237:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827807.510295:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827807.894231:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787827808.342299:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827808.342311:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787827808.342318:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827808.342331:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1694
00000100:00080000:0.0:1787827808.342346:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827808.342354:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787827808.342406:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626776192/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827808.342447:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827808.342453:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827808.342455:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827808.342460:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1694
00000100:00080000:0.0:1787827808.342464:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827808.342468:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827808.342508:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827808.342537:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626776320/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827808.342562:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626776192:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787827808.342571:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827808.342590:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787827808.342595:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827808.342605:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1694
00000100:00080000:0.0:1787827808.342635:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827808.342646:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827808.342657:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827808.342678:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626776448/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827808.342692:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827808.342698:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827808.342700:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827808.342705:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1694
00000100:00080000:0.0:1787827808.342711:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827808.342720:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827808.342733:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787827808.342735:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827808.342761:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626776576/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827808.342874:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827808/real 1787827808]  req@ffff8cd0c8e9e940 x1874671626776192/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827833 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827808.342914:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626776192/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827833 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827808.342938:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827808.342947:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827808.342955:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626776192:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827808.343006:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626776320:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827808.343059:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827808.343139:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827808/real 1787827808]  req@ffff8cd0c8e9c3c0 x1874671626776320/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827833 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827808.343170:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626776320/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827833 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827808.343186:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827808.343191:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827808.343220:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626776320:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827808.343238:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626776448:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827808.343289:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827808.343314:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827808/real 1787827808]  req@ffff8cd0c8e9ed00 x1874671626776448/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827833 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827808.343333:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626776448/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827833 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827808.343344:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827808.343348:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827808.343352:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626776448:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827808.343361:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626776576:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827808.343390:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827808.343404:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827808/real 1787827808]  req@ffff8cd0c8e9f840 x1874671626776576/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827833 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827808.343419:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626776576/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827833 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827808.343428:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827808.343432:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827808.343436:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626776576:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827808.534265:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827808.918235:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827809.558179:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827809.942234:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827810.582189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827810.966274:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827811.606161:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827811.990192:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827812.630251:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827813.014244:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787827813.462211:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827813.462224:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787827813.462229:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827813.462242:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1699
00000100:00080000:0.0:1787827813.462256:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827813.462265:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787827813.462313:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626776704/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827813.462349:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827813.462354:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827813.462358:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827813.462363:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1699
00000100:00080000:0.0:1787827813.462378:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827813.462386:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827813.462401:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827813.462419:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626776832/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827813.462435:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827813.462440:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787827813.462444:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827813.462451:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1699
00000100:00080000:0.0:1787827813.462460:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1787827813.462463:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626776704:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787827813.462488:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827813.462497:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827813.462529:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626776960/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827813.462547:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827813.462555:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827813.462559:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827813.462566:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1699
00000100:00080000:0.0:1787827813.462589:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1787827813.462600:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787827813.462608:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827813.462624:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827813.462668:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626777088/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827813.462711:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827813/real 1787827813]  req@ffff8cd0c8e9d680 x1874671626776704/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827838 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827813.462741:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626776704/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827838 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827813.462759:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827813.462766:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827813.462774:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626776704:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827813.462844:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626776832:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827813.462880:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827813.462898:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827813/real 1787827813]  req@ffff8cd0c8e9f480 x1874671626776832/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827838 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827813.462914:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626776832/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827838 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827813.462925:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827813.462930:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827813.462934:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626776832:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827813.462943:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626776960:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827813.462998:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827813.463012:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827813/real 1787827813]  req@ffff8cd0c8e9cb40 x1874671626776960/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827838 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827813.463027:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626776960/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827838 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827813.463037:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827813.463040:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827813.463044:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626776960:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827813.463052:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626777088:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827813.463073:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827813.463143:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827813/real 1787827813]  req@ffff8cd0c8e9c000 x1874671626777088/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827838 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827813.463161:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626777088/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827838 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827813.463172:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827813.463176:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827813.463180:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626777088:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827813.654225:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827814.038216:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827814.678220:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827815.062197:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827815.702191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827816.086259:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827816.726258:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827817.110263:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827817.750168:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827818.134240:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827818.582214:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827818.582227:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827818.582232:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827818.582244:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1704
00000100:00080000:1.0:1787827818.582256:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827818.582276:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827818.582336:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626777216/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827818.582385:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827818.582391:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827818.582394:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827818.582417:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1704
00000100:00080000:1.0:1787827818.582424:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827818.582431:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827818.582464:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827818.582498:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626777344/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827818.582513:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827818.582533:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827818.582536:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827818.582540:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1704
00000100:00080000:1.0:1787827818.582546:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1787827818.582554:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626777216:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787827818.582557:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827818.582565:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827818.582582:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626777472/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827818.582597:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827818.582601:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827818.582620:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827818.582627:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1704
00000100:00080000:1.0:1787827818.582632:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827818.582638:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827818.582644:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827818.582658:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626777600/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787827818.582695:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827818.582807:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827818/real 1787827818]  req@ffff8cd0d48f61c0 x1874671626777216/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827843 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827818.582833:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626777216/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827843 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827818.582845:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827818.582849:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827818.582871:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626777216:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827818.582879:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626777344:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827818.582903:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827818.582913:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827818/real 1787827818]  req@ffff8cd0d48f7c00 x1874671626777344/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827843 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827818.582940:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626777344/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827843 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827818.582947:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827818.582950:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827818.582952:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626777344:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827818.582983:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626777472:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827818.583022:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827818.583033:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827818/real 1787827818]  req@ffff8cd0d48f52c0 x1874671626777472/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827843 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827818.583062:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626777472/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827843 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827818.583071:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827818.583075:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827818.583078:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626777472:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827818.583091:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626777600:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827818.583112:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827818.583122:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827818/real 1787827818]  req@ffff8cd0d48f6d00 x1874671626777600/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827843 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827818.583133:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626777600/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827843 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827818.583206:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827818.583210:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827818.583212:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626777600:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827818.774370:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827819.158177:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827819.798168:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827820.182285:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827820.822239:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827821.206239:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827821.846191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827822.230209:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827822.870300:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827823.254230:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827823.702277:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827823.702291:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827823.702311:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827823.702323:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1709
00000100:00080000:1.0:1787827823.702336:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827823.702346:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827823.702403:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626777728/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827823.702440:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827823.702444:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827823.702447:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827823.702452:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1709
00000100:00080000:1.0:1787827823.702468:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827823.702473:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827823.702484:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827823.702499:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626777856/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827823.702515:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827823.702521:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827823.702525:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827823.702543:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1709
00000100:00080000:1.0:1787827823.702552:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827823.702561:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827823.702572:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827823.702597:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626777984/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827823.702638:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626777728:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787827823.702669:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827823.702675:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827823.702678:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827823.702684:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1709
00000100:00080000:1.0:1787827823.702690:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827823.702695:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827823.702703:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827823.702745:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626778112/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787827823.702803:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827823.702862:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827823/real 1787827823]  req@ffff8cd0d48f6d00 x1874671626777728/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827848 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827823.702899:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626777728/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827848 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827823.702912:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827823.702916:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827823.702921:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626777728:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827823.702930:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626777856:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827823.702956:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827823.702988:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827823/real 1787827823]  req@ffff8cd0d48f52c0 x1874671626777856/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827848 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827823.703005:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626777856/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827848 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827823.703017:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827823.703021:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827823.703026:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626777856:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827823.703072:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626777984:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827823.703120:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827823.703221:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827823/real 1787827823]  req@ffff8cd0d48f7c00 x1874671626777984/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827848 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827823.703240:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626777984/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827848 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827823.703285:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827823.703290:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827823.703296:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626777984:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827823.703309:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626778112:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827823.703352:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827823.703390:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827823/real 1787827823]  req@ffff8cd0d48f61c0 x1874671626778112/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827848 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827823.703409:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626778112/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827848 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827823.703419:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827823.703423:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827823.703427:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626778112:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827823.894273:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827824.278251:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827824.918253:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827825.302186:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827825.942341:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827826.326260:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827826.966223:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827827.350246:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827827.990227:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827828.374231:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827828.822222:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827828.822232:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827828.822236:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827828.822246:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1714
00000100:00080000:2.0:1787827828.822257:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827828.822264:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827828.822326:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626778240/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827828.822365:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827828.822368:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827828.822370:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827828.822374:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1714
00000100:00080000:2.0:1787827828.822378:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827828.822381:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827828.822390:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827828.822400:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab480 x1874671626778368/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827828.822421:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827828.822425:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827828.822426:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827828.822433:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1714
00000100:00100000:3.0:1787827828.822435:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626778240:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787827828.822437:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827828.822445:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827828.822450:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827828.822461:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626778496/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827828.822469:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827828.822473:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827828.822475:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827828.822485:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1714
00000100:00080000:2.0:1787827828.822498:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827828.822503:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827828.822508:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787827828.822514:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827828.822525:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626778624/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827828.822580:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827828/real 1787827828]  req@ffff8cd0c3da8780 x1874671626778240/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827853 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827828.822603:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626778240/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827853 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827828.822660:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827828.822667:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827828.822673:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626778240:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827828.822717:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626778368:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827828.822742:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827828.822755:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827828/real 1787827828]  req@ffff8cd0c3dab480 x1874671626778368/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827853 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827828.822766:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab480 x1874671626778368/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827853 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827828.822774:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827828.822776:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827828.822779:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626778368:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827828.822784:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626778496:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827828.822799:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827828.822807:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827828/real 1787827828]  req@ffff8cd0c3daa940 x1874671626778496/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827853 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827828.822816:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626778496/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827853 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827828.822823:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827828.822825:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827828.822827:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626778496:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827828.822840:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626778624:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827828.822857:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827828.822874:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827828/real 1787827828]  req@ffff8cd0c3dab0c0 x1874671626778624/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827853 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827828.822884:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626778624/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827853 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827828.822891:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827828.822893:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827828.822896:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626778624:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827829.014215:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827829.398228:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827830.038211:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827830.422282:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827831.062208:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827831.446235:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827832.086225:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827832.470220:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827833.110200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827833.494221:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827833.942216:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827833.942225:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827833.942229:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827833.942238:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1719
00000100:00080000:1.0:1787827833.942248:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827833.942256:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827833.942296:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626778752/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827833.942330:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827833.942333:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827833.942335:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827833.942338:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1719
00000100:00080000:1.0:1787827833.942342:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827833.942346:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827833.942371:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827833.942380:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626778752:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787827833.942391:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626778880/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827833.942403:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827833.942406:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827833.942407:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827833.942411:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1719
00000100:00080000:1.0:1787827833.942415:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827833.942418:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827833.942423:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827833.942430:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626779008/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827833.942437:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827833.942440:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827833.942441:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827833.942444:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1719
00000100:00080000:1.0:1787827833.942446:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827833.942448:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827833.942453:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827833.942459:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626779136/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787827833.942482:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827833.942533:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827833/real 1787827833]  req@ffff8cd0d48f70c0 x1874671626778752/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827858 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827833.942550:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626778752/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827858 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827833.942563:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827833.942568:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827833.942572:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626778752:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827833.942597:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626778880:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827833.942621:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827833.942662:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827833/real 1787827833]  req@ffff8cd0d48f43c0 x1874671626778880/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827858 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827833.942673:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626778880/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827858 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827833.942680:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827833.942682:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827833.942685:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626778880:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827833.942690:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626779008:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827833.942708:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827833.942716:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827833/real 1787827833]  req@ffff8cd0d48f5a40 x1874671626779008/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827858 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827833.942728:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626779008/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827858 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827833.942734:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827833.942737:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827833.942739:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626779008:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827833.942743:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626779136:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827833.942757:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827833.942763:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827833/real 1787827833]  req@ffff8cd0d48f7840 x1874671626779136/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827858 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827833.942771:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626779136/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827858 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827833.942782:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827833.942785:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827833.942789:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626779136:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827834.134156:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827834.518206:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827835.158181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827835.542265:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827836.182155:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827836.566245:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827837.206196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827837.590346:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827838.230152:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827838.614222:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827839.062214:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827839.062222:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827839.062225:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827839.062233:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1724
00000100:00080000:2.0:1787827839.062242:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827839.062248:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827839.062280:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626779264/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827839.062307:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827839.062310:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827839.062324:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827839.062327:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1724
00000100:00080000:2.0:1787827839.062332:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827839.062337:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827839.062341:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626779264:192.168.203.145@tcp:250:kworker.0
00000008:00100000:2.0:1787827839.062349:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827839.062367:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626779392/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827839.062382:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827839.062385:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827839.062387:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827839.062391:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1724
00000100:00080000:2.0:1787827839.062394:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827839.062399:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:3.0:1787827839.062399:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:2.0:1787827839.062407:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827839.062417:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626779520/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827839.062425:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827839.062427:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827839.062429:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827839.062432:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1724
00000100:00080000:2.0:1787827839.062436:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827839.062441:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827839.062444:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827839/real 1787827839]  req@ffff8cd0c3da9a40 x1874671626779264/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827864 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1787827839.062448:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827839.062474:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626779264/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827864 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827839.062476:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626779648/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827839.062488:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827839.062494:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827839.062499:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626779264:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827839.062519:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626779392:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827839.062535:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827839.062544:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827839/real 1787827839]  req@ffff8cd0c3da8f00 x1874671626779392/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827864 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827839.062553:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626779392/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827864 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827839.062558:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827839.062567:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827839.062569:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626779392:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827839.062574:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626779520:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827839.062589:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827839.062604:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827839/real 1787827839]  req@ffff8cd0c3da9680 x1874671626779520/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827864 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827839.062627:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626779520/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827864 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827839.062633:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827839.062635:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827839.062637:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626779520:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827839.062642:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626779648:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827839.062676:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827839.062685:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827839/real 1787827839]  req@ffff8cd0c3da83c0 x1874671626779648/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827864 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827839.062694:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626779648/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827864 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827839.062700:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827839.062703:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827839.062705:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626779648:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827839.254193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827839.638261:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827840.278206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827840.662182:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827841.302162:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827841.686185:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827842.326243:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827842.710219:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827843.350149:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827843.734297:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827844.182247:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827844.182258:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827844.182286:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827844.182296:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1730
00000100:00080000:1.0:1787827844.182309:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827844.182316:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827844.182359:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626779776/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827844.182393:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827844.182397:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827844.182399:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827844.182404:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1730
00000100:00080000:1.0:1787827844.182409:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827844.182422:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827844.182447:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827844.182450:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626779776:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787827844.182463:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626779904/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827844.182473:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827844.182477:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827844.182478:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827844.182481:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1730
00000100:00080000:1.0:1787827844.182484:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827844.182487:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827844.182491:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827844.182499:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626780032/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827844.182518:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827844.182520:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827844.182522:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827844.182526:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1730
00000100:00080000:1.0:1787827844.182529:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827844.182532:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827844.182536:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787827844.182537:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787827844.182549:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626780160/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827844.182611:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827844/real 1787827844]  req@ffff8cd0d48f4000 x1874671626779776/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827869 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827844.182655:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626779776/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827869 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827844.182669:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827844.182675:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827844.182680:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626779776:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827844.182707:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626779904:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827844.182728:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827844.182738:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827844/real 1787827844]  req@ffff8cd0d48f6580 x1874671626779904/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827869 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827844.182747:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626779904/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827869 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827844.182754:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827844.182756:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827844.182758:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626779904:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827844.182762:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626780032:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827844.182775:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827844.182790:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827844/real 1787827844]  req@ffff8cd0d48f7480 x1874671626780032/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827869 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827844.182798:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626780032/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827869 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827844.182804:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827844.182806:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827844.182808:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626780032:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827844.182812:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626780160:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827844.182835:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827844.182841:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827844/real 1787827844]  req@ffff8cd0d48f4b40 x1874671626780160/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827869 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827844.182849:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626780160/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827869 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827844.182854:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827844.182864:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827844.182866:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626780160:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827844.374209:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827844.758198:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827845.398238:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827845.782356:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827846.422237:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827846.806229:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827847.446195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827847.830226:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827848.470169:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827848.854180:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827849.302210:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827849.302227:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827849.302231:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827849.302243:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1735
00000100:00080000:1.0:1787827849.302257:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827849.302269:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827849.302335:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626780288/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827849.302363:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827849.302384:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827849.302388:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827849.302395:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1735
00000100:00080000:1.0:1787827849.302401:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827849.302408:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827849.302449:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626780288:192.168.203.145@tcp:250:kworker.0
00000008:00100000:1.0:1787827849.302458:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827849.302480:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626780416/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827849.302498:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827849.302503:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827849.302506:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827849.302511:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1735
00000100:00080000:1.0:1787827849.302518:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827849.302524:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827849.302531:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827849.302592:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626780544/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827849.302609:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827849.302614:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:3.0:1787827849.302620:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626780416:192.168.203.145@tcp:38:kworker.0
00000100:00080000:1.0:1787827849.302629:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827849.302636:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1735
00000100:00080000:1.0:1787827849.302641:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827849.302647:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827849.302655:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827849.302671:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626780544:192.168.203.145@tcp:8:kworker.0
00000100:00100000:1.0:1787827849.302673:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626780672/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827849.302737:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626780672:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827849.494303:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827849.878273:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827850.518207:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827850.902200:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827851.542221:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827851.926254:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:2.0:1787827852.374420:0:2343:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:2.0:1787827852.374472:0:2343:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:2.0:1787827852.374492:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787827852.374576:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787827852.374590:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787827852.374617:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:3.0:1787827852.374686:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827849/real 1787827852]  req@ffff8cd0d48f4000 x1874671626780672/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827874 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827852.374718:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626780672/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827874 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827852.374750:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827852.374758:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827852.374763:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626780672:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827852.374773:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827849/real 1787827852]  req@ffff8cd0d48f7480 x1874671626780416/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827874 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827852.374790:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626780416/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827874 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827852.374822:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827852.374827:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827852.374832:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626780416:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827852.374841:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827849/real 1787827852]  req@ffff8cd0d48f6580 x1874671626780544/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827874 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827852.374858:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626780544/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827874 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827852.374893:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827852.374898:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827852.374903:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626780544:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827852.374911:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827849/real 1787827852]  req@ffff8cd0d48f4b40 x1874671626780288/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827874 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827852.374928:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626780288/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827874 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827852.374938:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827852.374943:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827852.374947:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626780288:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827852.566142:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827852.950184:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827853.590286:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827853.974242:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827854.422297:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827854.422311:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827854.422316:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827854.422329:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1740
00000100:00080000:2.0:1787827854.422344:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827854.422354:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827854.422413:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8b40 x1874671626780800/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827854.422456:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827854.422460:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827854.422463:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827854.422468:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1740
00000100:00080000:2.0:1787827854.422473:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827854.422479:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827854.422492:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827854.422541:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626780928/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827854.422554:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626780800:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787827854.422557:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827854.422567:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827854.422569:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827854.422573:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1740
00000100:00080000:2.0:1787827854.422577:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827854.422581:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827854.422593:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827854.422609:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626781056/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827854.422621:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827854.422626:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827854.422628:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827854.422641:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1740
00000100:00080000:2.0:1787827854.422647:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1787827854.422652:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787827854.422657:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827854.422665:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827854.422682:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626781184/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827854.422734:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827854/real 1787827854]  req@ffff8cd0c3da8b40 x1874671626780800/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827879 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827854.422769:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8b40 x1874671626780800/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827879 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827854.422782:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827854.422787:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827854.422792:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626780800:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827854.422822:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626780928:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827854.422884:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827854.422902:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827854/real 1787827854]  req@ffff8cd0c3da83c0 x1874671626780928/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827879 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827854.422917:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626780928/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827879 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827854.422927:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827854.422931:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827854.422935:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626780928:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827854.422943:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626781056:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827854.422969:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827854.422983:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827854/real 1787827854]  req@ffff8cd0c3da9680 x1874671626781056/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827879 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827854.422996:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626781056/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827879 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827854.423113:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827854.423119:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827854.423123:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626781056:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827854.423130:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626781184:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827854.423156:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827854.423170:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827854/real 1787827854]  req@ffff8cd0c3da8f00 x1874671626781184/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827879 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827854.423201:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626781184/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827879 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827854.423210:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827854.423213:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827854.423217:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626781184:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827854.614189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827854.998251:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827855.638246:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827856.022267:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827856.662218:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827857.046296:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827857.686237:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827858.070287:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827858.710216:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827859.094213:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827859.542289:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827859.542300:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827859.542330:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827859.542341:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1745
00000100:00080000:1.0:1787827859.542366:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827859.542374:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827859.542419:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626781312/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827859.542448:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827859.542462:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827859.542477:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827859.542483:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1745
00000100:00080000:1.0:1787827859.542487:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827859.542492:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827859.542506:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827859.542519:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626781440/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827859.542529:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827859.542533:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827859.542535:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827859.542538:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1745
00000100:00080000:1.0:1787827859.542541:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827859.542544:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827859.542557:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827859.542565:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626781568/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827859.542580:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626781312:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787827859.542607:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827859.542612:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827859.542613:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827859.542617:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1745
00000100:00080000:1.0:1787827859.542622:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827859.542627:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827859.542632:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827859.542643:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626781696/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787827859.542697:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827859.542766:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827859/real 1787827859]  req@ffff8cd0d48f7840 x1874671626781312/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827884 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827859.542793:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626781312/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827884 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827859.542809:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827859.542815:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827859.542821:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626781312:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827859.542834:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626781440:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827859.542867:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827859.542884:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827859/real 1787827859]  req@ffff8cd0d48f5a40 x1874671626781440/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827884 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827859.542902:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626781440/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827884 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827859.542929:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827859.542935:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827859.542941:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626781440:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827859.542974:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626781568:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827859.543023:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827859.543041:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827859/real 1787827859]  req@ffff8cd0d48f43c0 x1874671626781568/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827884 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827859.543060:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626781568/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827884 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827859.543072:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827859.543077:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827859.543083:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626781568:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827859.543092:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626781696:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827859.543170:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827859.543225:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827859/real 1787827859]  req@ffff8cd0d48f70c0 x1874671626781696/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827884 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827859.543243:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626781696/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827884 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827859.543254:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827859.543274:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827859.543291:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626781696:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827859.734230:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827860.118246:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827860.758191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787827861.142218:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827861.782206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827862.166256:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827862.806214:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827863.190342:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827863.830257:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827864.214252:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827864.662160:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827864.662168:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827864.662172:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827864.662180:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1750
00000100:00080000:1.0:1787827864.662190:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827864.662195:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827864.662226:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626781824/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827864.662249:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827864.662251:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827864.662253:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827864.662255:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1750
00000100:00080000:1.0:1787827864.662257:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827864.662260:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827864.662283:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827864.662292:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626781952/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827864.662299:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827864.662302:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827864.662303:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827864.662305:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1750
00000100:00080000:1.0:1787827864.662308:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827864.662311:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827864.662315:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827864.662321:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626782080/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827864.662328:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827864.662331:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827864.662334:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1787827864.662335:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626781824:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787827864.662339:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1750
00000100:00080000:1.0:1787827864.662344:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827864.662347:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827864.662350:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827864.662359:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626782208/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787827864.662396:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827864.662453:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827864/real 1787827864]  req@ffff8cd0d48f61c0 x1874671626781824/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827889 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827864.662468:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626781824/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827889 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827864.662477:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827864.662479:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827864.662482:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626781824:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827864.662488:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626781952:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827864.662505:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827864.662514:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827864/real 1787827864]  req@ffff8cd0d48f7c00 x1874671626781952/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827889 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827864.662522:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626781952/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827889 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827864.662527:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827864.662529:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827864.662531:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626781952:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827864.662571:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626782080:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827864.662594:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827864.662606:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827864/real 1787827864]  req@ffff8cd0d48f52c0 x1874671626782080/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827889 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827864.662618:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626782080/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827889 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827864.662626:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827864.662629:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827864.662632:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626782080:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827864.662637:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626782208:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827864.662656:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827864.662666:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827864/real 1787827864]  req@ffff8cd0d48f6d00 x1874671626782208/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827889 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827864.662677:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626782208/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827889 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827864.662684:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827864.662687:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827864.662690:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626782208:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827864.854215:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827865.238278:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827865.878228:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827866.262214:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827866.902262:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827867.286272:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827867.926223:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827868.310214:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827868.950356:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827869.334362:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827869.782253:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827869.782271:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827869.782305:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827869.782322:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1755
00000100:00080000:1.0:1787827869.782340:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827869.782350:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827869.782428:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626782336/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827869.782469:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827869.782473:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827869.782476:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827869.782481:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1755
00000100:00080000:1.0:1787827869.782486:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827869.782491:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827869.782505:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827869.782525:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626782464/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827869.782544:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626782336:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787827869.782546:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827869.782556:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827869.782561:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827869.782567:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1755
00000100:00080000:1.0:1787827869.782571:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827869.782575:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827869.782595:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827869.782631:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626782592/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827869.782679:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827869.782690:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000800:00000100:3.0:1787827869.782693:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787827869.782696:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827869.782707:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1755
00000100:00080000:1.0:1787827869.782716:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827869.782723:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827869.782732:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827869.782754:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626782720/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827869.782770:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827869/real 1787827869]  req@ffff8cd0d48f6d00 x1874671626782336/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827894 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827869.782804:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626782336/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827894 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827869.782820:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827869.782826:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827869.782832:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626782336:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827869.782881:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626782464:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827869.782921:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827869.782938:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827869/real 1787827869]  req@ffff8cd0d48f52c0 x1874671626782464/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827894 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827869.782954:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626782464/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827894 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827869.782965:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827869.782969:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827869.782974:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626782464:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827869.782981:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626782592:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827869.783003:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827869.783015:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827869/real 1787827869]  req@ffff8cd0d48f7c00 x1874671626782592/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827894 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827869.783051:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626782592/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827894 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827869.783062:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827869.783066:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827869.783070:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626782592:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827869.783077:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626782720:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827869.783155:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827869.783170:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827869/real 1787827869]  req@ffff8cd0d48f61c0 x1874671626782720/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827894 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827869.783186:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626782720/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827894 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827869.783196:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827869.783200:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827869.783204:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626782720:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827869.974249:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827870.358327:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827870.998311:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827871.382324:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827872.022178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827872.406261:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827873.046169:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827873.430446:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827874.070293:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827874.454348:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827874.902316:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827874.902332:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827874.902340:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827874.902356:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1760
00000100:00080000:1.0:1787827874.902402:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827874.902414:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827874.902488:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626782848/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827874.902537:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827874.902545:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827874.902549:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827874.902556:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1760
00000100:00080000:1.0:1787827874.902564:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827874.902575:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827874.902611:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626782848:192.168.203.145@tcp:250:kworker.0
00000008:00100000:1.0:1787827874.902621:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827874.902674:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626782976/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827874.902703:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827874.902714:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827874.902718:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:3.0:1787827874.902721:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787827874.902731:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1760
00000100:00080000:1.0:1787827874.902744:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827874.902755:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827874.902768:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827874.902786:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827874/real 1787827874]  req@ffff8cd0d48f70c0 x1874671626782848/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827899 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787827874.902797:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626783104/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827874.902827:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827874.902847:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827874.902856:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827874.902876:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1760
00000100:00080000:1.0:1787827874.902884:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827874.902893:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827874.902898:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626782848/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827899 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1787827874.902907:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00080000:3.0:1787827874.902920:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827874.902929:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827874.902941:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626782848:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787827874.902943:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626783232/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827874.902979:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626782976:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827874.903031:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827874.903052:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827874/real 1787827874]  req@ffff8cd0d48f43c0 x1874671626782976/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827899 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827874.903068:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626782976/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827899 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827874.903141:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827874.903146:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827874.903151:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626782976:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827874.903160:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626783104:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827874.903190:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827874.903214:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827874/real 1787827874]  req@ffff8cd0d48f5a40 x1874671626783104/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827899 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827874.903230:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626783104/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827899 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827874.903259:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827874.903264:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827874.903268:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626783104:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827874.903276:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626783232:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827874.903330:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827874.903345:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827874/real 1787827874]  req@ffff8cd0d48f7840 x1874671626783232/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827899 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827874.903383:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626783232/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827899 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827874.903394:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827874.903398:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827874.903403:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626783232:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827875.094299:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827875.478392:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827876.118282:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827876.502300:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827877.142227:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827877.526508:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827878.166321:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827878.550313:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827879.190238:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827879.574246:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827880.022191:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827880.022199:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827880.022203:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827880.022211:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1765
00000100:00080000:2.0:1787827880.022221:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827880.022227:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827880.022258:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626783360/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827880.022281:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827880.022283:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827880.022285:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827880.022287:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1765
00000100:00080000:2.0:1787827880.022290:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827880.022292:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827880.022299:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827880.022307:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626783488/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827880.022344:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626783360:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787827880.022353:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827880.022356:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827880.022358:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827880.022362:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1765
00000100:00080000:2.0:1787827880.022366:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827880.022370:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827880.022375:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827880.022387:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626783616/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827880.022394:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827880.022397:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827880.022398:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827880.022400:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1765
00000100:00080000:2.0:1787827880.022403:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1787827880.022408:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787827880.022409:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827880.022415:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827880.022424:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab480 x1874671626783744/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827880.022447:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827880/real 1787827880]  req@ffff8cd0c3da9a40 x1874671626783360/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827905 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827880.022462:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626783360/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827905 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827880.022470:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827880.022479:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827880.022482:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626783360:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827880.022502:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626783488:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827880.022520:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827880.022529:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827880/real 1787827880]  req@ffff8cd0c3dab0c0 x1874671626783488/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827905 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827880.022537:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626783488/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827905 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827880.022543:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827880.022545:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827880.022547:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626783488:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827880.022551:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626783616:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827880.022563:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827880.022569:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827880/real 1787827880]  req@ffff8cd0c3daa940 x1874671626783616/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827905 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827880.022577:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626783616/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827905 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827880.022584:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827880.022587:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827880.022591:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626783616:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827880.022597:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626783744:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827880.022628:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827880.022641:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827880/real 1787827880]  req@ffff8cd0c3dab480 x1874671626783744/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827905 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827880.022653:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab480 x1874671626783744/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827905 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827880.022658:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827880.022661:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827880.022663:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626783744:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827880.214172:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827880.598245:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827881.238159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827881.622203:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827882.262148:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827882.646249:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827883.286167:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827883.670205:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827884.310167:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827884.694204:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827885.142178:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827885.142186:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827885.142202:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827885.142210:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1770
00000100:00080000:1.0:1787827885.142219:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827885.142225:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827885.142254:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626783872/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827885.142278:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827885.142280:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827885.142282:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827885.142285:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1770
00000100:00080000:1.0:1787827885.142298:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827885.142302:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827885.142311:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827885.142322:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626783872:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787827885.142323:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626784000/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827885.142338:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827885.142341:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827885.142342:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827885.142346:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1770
00000100:00080000:1.0:1787827885.142349:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827885.142351:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827885.142354:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827885.142364:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626784128/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827885.142382:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827885.142387:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000800:00000100:3.0:1787827885.142387:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787827885.142391:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827885.142396:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1770
00000100:00080000:1.0:1787827885.142401:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827885.142404:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827885.142408:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827885.142418:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626784256/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827885.142436:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827885/real 1787827885]  req@ffff8cd0d48f4b40 x1874671626783872/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827910 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827885.142458:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626783872/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827910 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827885.142470:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827885.142475:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827885.142479:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626783872:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827885.142502:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626784000:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827885.142528:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827885.142550:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827885/real 1787827885]  req@ffff8cd0d48f6580 x1874671626784000/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827910 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827885.142568:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626784000/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827910 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827885.142575:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827885.142578:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827885.142580:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626784000:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827885.142585:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626784128:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827885.142601:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827885.142609:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827885/real 1787827885]  req@ffff8cd0d48f7480 x1874671626784128/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827910 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827885.142617:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626784128/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827910 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827885.142622:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827885.142625:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827885.142627:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626784128:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827885.142630:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626784256:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827885.142640:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827885.142646:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827885/real 1787827885]  req@ffff8cd0d48f4000 x1874671626784256/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827910 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827885.142653:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626784256/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827910 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827885.142664:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827885.142666:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827885.142669:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626784256:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827885.334168:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827885.718204:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827886.358173:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827886.742237:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827887.383257:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827887.766325:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827888.406195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827888.790259:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827889.430166:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827889.814215:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787827890.262212:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827890.262219:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787827890.262223:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827890.262231:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1776
00000100:00080000:0.0:1787827890.262241:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827890.262246:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787827890.262295:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d2c0 x1874671626784384/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827890.262319:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827890.262340:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827890.262342:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827890.262347:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1776
00000100:00080000:0.0:1787827890.262352:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827890.262357:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827890.262361:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626784384:192.168.203.145@tcp:250:kworker.0
00000008:00100000:0.0:1787827890.262366:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827890.262378:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626784512/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827890.262389:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827890.262392:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787827890.262394:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827890.262397:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1776
00000100:00080000:0.0:1787827890.262400:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827890.262404:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827890.262408:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827890.262417:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626784640/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787827890.262419:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787827890.262426:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827890.262429:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827890.262431:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827890.262434:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1776
00000100:00080000:0.0:1787827890.262437:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827890.262441:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827890.262451:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827890.262462:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626784768/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827890.262466:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827890/real 1787827890]  req@ffff8cd0c8e9d2c0 x1874671626784384/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827915 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827890.262488:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d2c0 x1874671626784384/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827915 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827890.262501:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827890.262506:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827890.262510:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626784384:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827890.262534:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626784512:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827890.262554:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827890.262563:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827890/real 1787827890]  req@ffff8cd0c8e9c000 x1874671626784512/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827915 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827890.262572:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626784512/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827915 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827890.262577:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827890.262586:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827890.262590:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626784512:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827890.262596:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626784640:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827890.262627:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827890.262642:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827890/real 1787827890]  req@ffff8cd0c8e9cb40 x1874671626784640/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827915 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827890.262650:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626784640/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827915 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827890.262656:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827890.262658:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827890.262660:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626784640:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827890.262664:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626784768:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827890.262677:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827890.262688:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827890/real 1787827890]  req@ffff8cd0c8e9f480 x1874671626784768/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827915 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827890.262697:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626784768/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827915 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827890.262703:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827890.262705:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827890.262707:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626784768:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827890.454176:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827890.838244:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827891.478170:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827891.862197:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827892.502190:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827892.887606:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827893.526159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827893.910319:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827894.550202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827894.934202:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827895.382243:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827895.382257:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827895.382262:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827895.382274:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1781
00000100:00080000:1.0:1787827895.382289:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827895.382296:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827895.382348:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626784896/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827895.382381:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827895.382385:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827895.382387:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827895.382391:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1781
00000100:00080000:1.0:1787827895.382395:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827895.382400:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827895.382432:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827895.382436:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626784896:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787827895.382452:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626785024/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827895.382466:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827895.382471:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827895.382473:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827895.382477:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1781
00000100:00080000:1.0:1787827895.382482:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827895.382486:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827895.382492:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827895.382504:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626785152/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827895.382514:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:3.0:1787827895.382520:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787827895.382521:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827895.382526:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827895.382530:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1781
00000100:00080000:1.0:1787827895.382534:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827895.382538:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827895.382543:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827895.382554:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626785280/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827895.382583:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827895/real 1787827895]  req@ffff8cd0d48f4000 x1874671626784896/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827920 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827895.382600:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626784896/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827920 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827895.382610:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827895.382613:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827895.382617:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626784896:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827895.382640:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626785024:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827895.382659:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827895.382669:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827895/real 1787827895]  req@ffff8cd0d48f7480 x1874671626785024/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827920 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827895.382677:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626785024/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827920 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827895.382683:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827895.382685:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827895.382687:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626785024:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827895.382691:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626785152:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827895.382703:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827895.382719:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827895/real 1787827895]  req@ffff8cd0d48f6580 x1874671626785152/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827920 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827895.382727:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626785152/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827920 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827895.382733:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827895.382735:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827895.382737:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626785152:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827895.382740:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626785280:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827895.382752:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827895.382758:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827895/real 1787827895]  req@ffff8cd0d48f4b40 x1874671626785280/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827920 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827895.382765:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626785280/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827920 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827895.382770:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827895.382772:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827895.382774:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626785280:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827895.574166:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827895.958283:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827896.598180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827896.982230:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827897.622159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827898.006251:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827898.646170:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827899.030252:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827899.670135:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827900.054235:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787827900.502231:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827900.502246:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787827900.502251:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827900.502263:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1786
00000100:00080000:0.0:1787827900.502276:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827900.502285:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787827900.502331:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626785408/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827900.502364:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827900.502368:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827900.502370:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827900.502374:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1786
00000100:00080000:0.0:1787827900.502379:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827900.502384:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827900.502396:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827900.502411:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626785536/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827900.502425:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787827900.502430:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626785408:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787827900.502431:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787827900.502438:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827900.502445:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1786
00000100:00080000:0.0:1787827900.502471:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827900.502478:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827900.502487:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787827900.502501:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787827900.502508:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626785664/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827900.502523:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827900.502527:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827900.502530:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827900.502537:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1786
00000100:00080000:0.0:1787827900.502542:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827900.502548:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827900.502550:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827900/real 1787827900]  req@ffff8cd0c8e9d680 x1874671626785408/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827925 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:0.0:1787827900.502557:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827900.502576:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626785792/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827900.502594:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626785408/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827925 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827900.502608:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827900.502613:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827900.502617:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626785408:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827900.502644:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626785536:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827900.502663:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827900.502681:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827900/real 1787827900]  req@ffff8cd0c8e9f840 x1874671626785536/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827925 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827900.502693:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626785536/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827925 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827900.502698:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827900.502701:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827900.502703:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626785536:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827900.502707:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626785664:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827900.502720:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827900.502726:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827900/real 1787827900]  req@ffff8cd0c8e9ed00 x1874671626785664/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827925 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827900.502734:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626785664/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827925 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827900.502748:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827900.502750:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827900.502753:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626785664:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827900.502760:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626785792:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827900.502783:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827900.502796:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827900/real 1787827900]  req@ffff8cd0c8e9c3c0 x1874671626785792/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827925 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827900.502810:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626785792/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827925 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827900.502819:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827900.502821:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827900.502824:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626785792:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827900.694150:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827901.078230:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827901.718143:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827902.102286:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827902.742199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827903.126267:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827903.766190:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827904.150289:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827904.790158:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827905.174270:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787827905.622166:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827905.622177:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787827905.622181:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827905.622192:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1791
00000100:00080000:0.0:1787827905.622205:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827905.622213:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787827905.622263:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626785920/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827905.622292:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827905.622312:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827905.622314:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827905.622318:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1791
00000100:00080000:0.0:1787827905.622322:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827905.622326:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827905.622336:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827905.622350:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626786048/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827905.622364:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787827905.622370:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626785920:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787827905.622371:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787827905.622377:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827905.622388:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1791
00000100:00080000:0.0:1787827905.622393:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827905.622398:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827905.622403:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827905.622415:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626786176/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787827905.622423:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787827905.622425:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787827905.622427:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787827905.622430:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1791
00000100:00080000:0.0:1787827905.622434:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787827905.622436:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787827905.622447:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787827905.622456:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626786304/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787827905.622465:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827905.622519:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827905/real 1787827905]  req@ffff8cd0c8e9e940 x1874671626785920/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827930 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827905.622535:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626785920/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827930 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827905.622561:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827905.622564:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827905.622567:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626785920:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827905.622589:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626786048:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827905.622614:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827905.622629:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827905/real 1787827905]  req@ffff8cd0c8e9e580 x1874671626786048/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827930 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827905.622643:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626786048/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827930 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827905.622652:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827905.622656:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827905.622659:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626786048:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827905.622664:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626786176:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827905.622682:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827905.622693:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827905/real 1787827905]  req@ffff8cd0c8e9de00 x1874671626786176/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827930 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827905.622706:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626786176/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827930 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827905.622715:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827905.622719:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827905.622722:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626786176:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827905.622737:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626786304:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827905.622756:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827905.622765:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827905/real 1787827905]  req@ffff8cd0c8e9cf00 x1874671626786304/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827930 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827905.622775:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626786304/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827930 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827905.622783:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827905.622787:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827905.622791:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626786304:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827905.814211:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827906.198312:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827906.838214:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827907.222249:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827907.862218:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827908.246241:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827908.886174:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827909.270270:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827909.910196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827910.294308:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827910.742200:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827910.742211:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827910.742215:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827910.742227:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1796
00000100:00080000:2.0:1787827910.742242:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827910.742250:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827910.742297:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626786432/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827910.742328:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827910.742332:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827910.742334:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827910.742338:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1796
00000100:00080000:2.0:1787827910.742342:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827910.742346:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827910.742358:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827910.742371:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626786560/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827910.742383:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827910.742389:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:3.0:1787827910.742389:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626786432:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787827910.742394:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827910.742401:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1796
00000100:00080000:2.0:1787827910.742403:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827910.742424:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827910.742428:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827910.742441:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626786688/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827910.742451:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827910.742455:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827910.742458:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827910.742462:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1796
00000100:00080000:2.0:1787827910.742467:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827910.742472:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827910.742478:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787827910.742483:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827910.742495:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626786816/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827910.742541:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827910/real 1787827910]  req@ffff8cd0c3da8780 x1874671626786432/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827935 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827910.742567:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626786432/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827935 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827910.742577:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827910.742587:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827910.742590:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626786432:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827910.742617:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626786560:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827910.742638:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827910.742647:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827910/real 1787827910]  req@ffff8cd0c3dabc00 x1874671626786560/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827935 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827910.742656:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626786560/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827935 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827910.742662:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827910.742664:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827910.742667:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626786560:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827910.742671:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626786688:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827910.742683:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827910.742692:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827910/real 1787827910]  req@ffff8cd0c3da8000 x1874671626786688/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827935 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827910.742704:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626786688/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827935 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827910.742713:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827910.742716:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827910.742720:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626786688:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827910.742727:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626786816:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827910.742757:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827910.742770:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827910/real 1787827910]  req@ffff8cd0c3dab840 x1874671626786816/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827935 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827910.742783:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626786816/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827935 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827910.742791:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827910.742794:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827910.742798:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626786816:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827910.934200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827911.318238:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827911.958209:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827912.342310:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827912.982178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827913.366298:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827914.006146:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827914.390290:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827915.030146:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827915.414280:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827915.862202:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827915.862216:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827915.862222:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827915.862236:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1801
00000100:00080000:2.0:1787827915.862250:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827915.862259:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827915.862309:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626786944/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827915.862345:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827915.862373:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827915.862376:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827915.862382:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1801
00000100:00080000:2.0:1787827915.862387:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827915.862393:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827915.862406:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827915.862410:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626786944:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827915.862428:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626787072/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827915.862443:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827915.862447:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827915.862450:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827915.862454:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1801
00000100:00080000:2.0:1787827915.862476:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827915.862482:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827915.862489:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827915.862503:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626787200/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827915.862515:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827915.862519:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827915.862521:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827915.862526:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1801
00000100:00080000:2.0:1787827915.862531:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827915.862536:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827915.862552:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827915.862554:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626787072:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827915.862572:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626787328/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827915.862596:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626787200:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827915.862632:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626787328:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827916.054201:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827916.438235:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827917.078162:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827917.462283:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827918.102180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827918.486221:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:1.0:1787827918.934407:0:2345:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:1.0:1787827918.934462:0:2345:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1787827918.934515:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827918.934552:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827918.934560:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827918.934567:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:3.0:1787827918.934614:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827915/real 1787827918]  req@ffff8cd0c3daa580 x1874671626787328/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827940 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827918.934647:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626787328/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827940 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827918.934661:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827918.934666:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827918.934671:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626787328:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827918.934680:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827915/real 1787827918]  req@ffff8cd0c3da9e00 x1874671626787072/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827940 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827918.934695:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626787072/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827940 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827918.934703:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827918.934706:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827918.934709:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626787072:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827918.934716:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827915/real 1787827918]  req@ffff8cd0c3da92c0 x1874671626787200/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827940 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827918.934727:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626787200/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827940 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827918.934735:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827918.934738:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827918.934741:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626787200:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827918.934757:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827915/real 1787827918]  req@ffff8cd0c3daa1c0 x1874671626786944/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827940 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827918.934772:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626786944/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827940 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827918.934781:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827918.934786:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827918.934789:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626786944:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827919.126148:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827919.510207:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827920.150254:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827920.534219:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827920.982196:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827920.982204:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827920.982208:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827920.982216:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1806
00000100:00080000:1.0:1787827920.982225:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827920.982231:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827920.982259:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626787456/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827920.982280:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827920.982283:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827920.982284:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827920.982287:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1806
00000100:00080000:1.0:1787827920.982289:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827920.982292:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827920.982301:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827920.982314:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626787584/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827920.982326:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827920.982330:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827920.982332:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1787827920.982340:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626787456:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787827920.982351:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1806
00000100:00080000:1.0:1787827920.982357:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827920.982362:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827920.982369:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827920.982384:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626787712/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827920.982396:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827920.982399:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827920.982401:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:3.0:1787827920.982406:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787827920.982407:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1806
00000100:00080000:1.0:1787827920.982414:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827920.982420:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827920.982425:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827920.982439:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626787840/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827920.982445:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827920/real 1787827920]  req@ffff8cd0d48f7840 x1874671626787456/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827945 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827920.982465:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626787456/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827945 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827920.982476:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827920.982481:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827920.982485:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626787456:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827920.982493:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626787584:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827920.982517:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827920.982537:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827920/real 1787827920]  req@ffff8cd0d48f5a40 x1874671626787584/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827945 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827920.982574:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626787584/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827945 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827920.982584:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827920.982588:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827920.982592:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626787584:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827920.982622:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626787712:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827920.982648:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827920.982662:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827920/real 1787827920]  req@ffff8cd0d48f43c0 x1874671626787712/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827945 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827920.982676:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626787712/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827945 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827920.982686:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827920.982689:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827920.982693:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626787712:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827920.982699:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626787840:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827920.982720:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827920.982733:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827920/real 1787827920]  req@ffff8cd0d48f70c0 x1874671626787840/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827945 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827920.982746:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626787840/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827945 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827920.982769:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827920.982772:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827920.982775:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626787840:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827921.174157:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827921.558242:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827922.198174:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827922.582188:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827923.222182:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827923.606250:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827924.246208:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827924.630279:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827925.270231:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827925.654308:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827926.102184:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827926.102196:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827926.102212:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827926.102232:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1811
00000100:00080000:1.0:1787827926.102244:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827926.102253:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827926.102295:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626787968/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827926.102325:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827926.102343:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827926.102345:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827926.102349:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1811
00000100:00080000:1.0:1787827926.102355:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827926.102361:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827926.102373:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827926.102404:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626788096/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827926.102419:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787827926.102422:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626787968:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787827926.102424:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827926.102429:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827926.102439:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1811
00000100:00080000:1.0:1787827926.102443:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827926.102446:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827926.102451:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827926.102484:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626788224/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827926.102495:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827926.102499:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827926.102501:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827926.102505:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1811
00000100:00080000:1.0:1787827926.102519:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827926.102525:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:3.0:1787827926.102530:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:1.0:1787827926.102543:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827926.102572:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626788352/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827926.102607:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827926/real 1787827926]  req@ffff8cd0d48f61c0 x1874671626787968/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827951 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827926.102630:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626787968/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827951 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827926.102643:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827926.102647:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827926.102652:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626787968:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827926.102684:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626788096:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827926.102736:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827926.102756:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827926/real 1787827926]  req@ffff8cd0d48f7c00 x1874671626788096/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827951 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827926.102794:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626788096/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827951 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827926.102805:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827926.102809:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827926.102812:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626788096:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827926.102819:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626788224:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827926.102846:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827926.102884:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827926/real 1787827926]  req@ffff8cd0d48f52c0 x1874671626788224/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827951 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827926.102898:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626788224/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827951 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827926.102906:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827926.102909:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827926.102913:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626788224:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827926.102918:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626788352:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827926.102939:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827926.102948:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827926/real 1787827926]  req@ffff8cd0d48f6d00 x1874671626788352/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827951 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827926.102959:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626788352/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827951 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827926.102982:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827926.102986:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827926.102990:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626788352:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827926.294184:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827926.678297:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827927.318250:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827927.702254:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827928.342177:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827928.726329:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827929.366193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827929.750255:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827930.390178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827930.774235:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787827931.222205:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827931.222218:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787827931.222226:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827931.222242:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1817
00000100:00080000:1.0:1787827931.222260:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827931.222275:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787827931.222337:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626788480/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827931.222388:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827931.222396:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827931.222400:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827931.222406:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1817
00000100:00080000:1.0:1787827931.222412:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827931.222419:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827931.222433:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827931.222455:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626788608/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827931.222475:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626788480:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787827931.222476:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827931.222490:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787827931.222498:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787827931.222504:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1817
00000100:00080000:1.0:1787827931.222531:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827931.222537:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827931.222546:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827931.222564:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626788736/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787827931.222578:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787827931.222583:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787827931.222586:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:3.0:1787827931.222595:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787827931.222603:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1817
00000100:00080000:1.0:1787827931.222611:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787827931.222617:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787827931.222625:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787827931.222641:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626788864/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827931.222663:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827931/real 1787827931]  req@ffff8cd0d48f6d00 x1874671626788480/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827956 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827931.222707:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626788480/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827956 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827931.222722:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827931.222728:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827931.222733:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626788480:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827931.222769:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626788608:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827931.222817:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827931.222844:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827931/real 1787827931]  req@ffff8cd0d48f52c0 x1874671626788608/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827956 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827931.222890:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626788608/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827956 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827931.222901:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827931.222906:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827931.222910:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626788608:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827931.222919:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626788736:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827931.222952:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827931.222992:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827931/real 1787827931]  req@ffff8cd0d48f7c00 x1874671626788736/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827956 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827931.223011:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626788736/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827956 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827931.223022:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827931.223026:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827931.223030:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626788736:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827931.223039:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626788864:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827931.223067:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827931.223081:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827931/real 1787827931]  req@ffff8cd0d48f61c0 x1874671626788864/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827956 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827931.223135:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626788864/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827956 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827931.223164:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827931.223169:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827931.223173:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626788864:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827931.414176:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827931.798294:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827932.438223:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827932.822274:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827933.462224:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827933.846280:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827934.486188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827934.870292:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827935.510258:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827935.894319:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827936.342199:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827936.342207:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827936.342211:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827936.342220:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1822
00000100:00080000:2.0:1787827936.342229:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827936.342235:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827936.342276:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daad00 x1874671626788992/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827936.342307:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827936.342310:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827936.342312:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827936.342315:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1822
00000100:00080000:2.0:1787827936.342318:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827936.342336:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827936.342350:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827936.342362:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626789120/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827936.342372:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827936.342375:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827936.342377:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827936.342380:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1822
00000100:00080000:2.0:1787827936.342383:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827936.342385:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827936.342389:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827936.342392:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626788992:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827936.342399:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626789248/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827936.342408:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827936.342411:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827936.342412:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827936.342415:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1822
00000100:00080000:2.0:1787827936.342417:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827936.342420:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827936.342424:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827936.342432:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626789376/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787827936.342459:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827936.342512:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827936/real 1787827936]  req@ffff8cd0c3daad00 x1874671626788992/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827961 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827936.342529:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daad00 x1874671626788992/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827961 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827936.342539:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827936.342542:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827936.342545:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626788992:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827936.342551:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626789120:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827936.342570:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827936.342580:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827936/real 1787827936]  req@ffff8cd0c3daa1c0 x1874671626789120/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827961 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827936.342614:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626789120/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827961 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827936.342622:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827936.342625:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827936.342628:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626789120:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827936.342662:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626789248:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827936.342684:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827936.342694:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827936/real 1787827936]  req@ffff8cd0c3da92c0 x1874671626789248/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827961 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827936.342704:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626789248/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827961 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827936.342711:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827936.342714:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827936.342717:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626789248:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827936.342737:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626789376:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827936.342748:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827936.342755:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827936/real 1787827936]  req@ffff8cd0c3da9e00 x1874671626789376/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827961 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827936.342767:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626789376/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827961 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827936.342788:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827936.342792:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827936.342796:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626789376:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827936.534179:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827936.918252:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827937.558203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827937.942274:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827938.582229:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827938.966302:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827939.606141:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827939.990212:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827940.630158:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827941.014271:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827941.462244:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827941.462258:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827941.462264:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827941.462277:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1827
00000100:00080000:2.0:1787827941.462291:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827941.462300:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827941.462350:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626789504/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827941.462390:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827941.462397:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827941.462400:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827941.462407:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1827
00000100:00080000:2.0:1787827941.462414:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827941.462421:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827941.462437:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827941.462474:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626789504:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827941.462485:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626789632/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827941.462505:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827941.462511:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827941.462515:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827941.462521:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1827
00000100:00080000:2.0:1787827941.462527:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827941.462532:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827941.462564:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787827941.462575:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827941.462583:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626789760/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827941.462600:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827941.462606:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827941.462610:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827941.462632:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1827
00000100:00080000:2.0:1787827941.462640:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827941.462648:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827941.462656:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827941.462658:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827941/real 1787827941]  req@ffff8cd0c3daa580 x1874671626789504/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827966 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827941.462674:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626789888/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827941.462690:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626789504/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827966 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827941.462708:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827941.462717:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827941.462724:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626789504:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827941.462764:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626789632:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827941.462806:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827941.462839:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827941/real 1787827941]  req@ffff8cd0c3dab840 x1874671626789632/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827966 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827941.462874:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626789632/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827966 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827941.462884:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827941.462888:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827941.462893:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626789632:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827941.462927:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626789760:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827941.462959:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827941.462973:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827941/real 1787827941]  req@ffff8cd0c3da8000 x1874671626789760/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827966 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827941.462987:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626789760/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827966 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827941.462997:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827941.463001:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827941.463004:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626789760:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827941.463011:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626789888:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827941.463028:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827941.463039:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827941/real 1787827941]  req@ffff8cd0c3dabc00 x1874671626789888/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827966 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827941.463052:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626789888/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827966 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827941.463076:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827941.463081:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827941.463122:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626789888:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827941.654224:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827942.038210:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827942.678192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827943.062251:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827943.702234:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827944.086286:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827944.726191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827945.110279:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827945.750184:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827946.134224:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827946.582189:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827946.582201:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827946.582205:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827946.582217:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1832
00000100:00080000:2.0:1787827946.582232:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827946.582240:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827946.582292:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626790016/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827946.582318:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827946.582321:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827946.582340:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827946.582345:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1832
00000100:00080000:2.0:1787827946.582349:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827946.582353:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827946.582369:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827946.582385:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab480 x1874671626790144/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827946.582392:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626790016:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787827946.582400:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827946.582405:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827946.582407:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827946.582411:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1832
00000100:00080000:2.0:1787827946.582413:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827946.582417:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827946.582425:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827946.582435:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626790272/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827946.582443:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827946.582446:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827946.582448:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827946.582453:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1832
00000100:00080000:2.0:1787827946.582463:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827946.582467:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827946.582473:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787827946.582477:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827946.582499:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626790400/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827946.582543:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827946/real 1787827946]  req@ffff8cd0c3da8780 x1874671626790016/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827971 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827946.582567:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626790016/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827971 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827946.582579:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827946.582586:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827946.582592:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626790016:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827946.582626:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626790144:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827946.582663:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827946.582679:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827946/real 1787827946]  req@ffff8cd0c3dab480 x1874671626790144/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827971 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827946.582708:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab480 x1874671626790144/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827971 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827946.582717:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827946.582721:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827946.582724:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626790144:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827946.582731:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626790272:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827946.582755:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827946.582765:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827946/real 1787827946]  req@ffff8cd0c3daa940 x1874671626790272/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827971 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827946.582776:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626790272/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827971 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827946.582783:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827946.582786:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827946.582789:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626790272:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827946.582795:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626790400:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827946.582810:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827946.582819:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827946/real 1787827946]  req@ffff8cd0c3dab0c0 x1874671626790400/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827971 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827946.582830:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626790400/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827971 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827946.582850:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827946.582854:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827946.582858:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626790400:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827946.774183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827947.158402:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827947.798155:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827948.182220:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827948.822200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827949.206298:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827949.846191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827950.230297:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827950.870182:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827951.254278:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827951.702298:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827951.702315:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827951.702322:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827951.702339:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1837
00000100:00080000:2.0:1787827951.702355:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827951.702364:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827951.702413:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626790528/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827951.702451:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827951.702457:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827951.702460:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827951.702467:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1837
00000100:00080000:2.0:1787827951.702472:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827951.702478:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827951.702493:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827951.702514:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626790656/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827951.702528:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827951.702531:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827951.702556:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1787827951.702565:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626790528:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787827951.702572:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1837
00000100:00080000:2.0:1787827951.702582:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827951.702591:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827951.702602:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787827951.702646:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827951.702652:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626790784/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827951.702672:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827951.702678:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827951.702682:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1787827951.702687:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827951/real 1787827951]  req@ffff8cd0c3da9a40 x1874671626790528/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827976 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827951.702696:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1837
00000100:00080000:2.0:1787827951.702714:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1787827951.702730:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626790528/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827976 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827951.702751:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00080000:3.0:1787827951.702752:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827951.702763:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827951.702769:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626790528:192.168.203.145@tcp:250:kworker.0
00000008:00100000:2.0:1787827951.702773:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827951.702778:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626790656:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827951.702805:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626790912/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787827951.702818:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827951.702853:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827951/real 1787827951]  req@ffff8cd0c3da8f00 x1874671626790656/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827976 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827951.702937:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626790656/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827976 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827951.702945:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827951.702948:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827951.702952:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626790656:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827951.702979:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626790784:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827951.702996:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827951.703003:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827951/real 1787827951]  req@ffff8cd0c3da9680 x1874671626790784/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827976 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827951.703012:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626790784/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827976 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827951.703027:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827951.703029:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827951.703031:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626790784:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827951.703035:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626790912:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827951.703046:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827951.703051:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827951/real 1787827951]  req@ffff8cd0c3da83c0 x1874671626790912/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827976 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827951.703059:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626790912/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827976 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827951.703073:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827951.703075:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827951.703078:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626790912:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827951.894156:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827952.278278:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827952.918197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827953.302318:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827953.942222:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827954.326411:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827954.966195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827955.350254:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827955.990178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827956.374264:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827956.822247:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827956.822266:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827956.822273:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827956.822290:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1842
00000100:00080000:2.0:1787827956.822348:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827956.822361:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827956.822479:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8b40 x1874671626791040/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827956.822535:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827956.822541:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827956.822545:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827956.822553:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1842
00000100:00080000:2.0:1787827956.822561:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827956.822570:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827956.822580:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626791040:192.168.203.145@tcp:250:kworker.0
00000008:00100000:2.0:1787827956.822593:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827956.822641:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626791168/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827956.822664:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:3.0:1787827956.822665:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787827956.822679:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827956.822683:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827956.822690:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1842
00000100:00080000:2.0:1787827956.822698:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827956.822704:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827956.822711:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827956.822724:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827956/real 1787827956]  req@ffff8cd0c3da8b40 x1874671626791040/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827981 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827956.822733:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626791296/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827956.822757:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8b40 x1874671626791040/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827981 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827956.822759:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827956.822777:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827956.822804:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827956.822810:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00080000:2.0:1787827956.822825:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1787827956.822825:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626791040:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787827956.822848:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1842
00000100:00100000:3.0:1787827956.822852:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626791168:192.168.203.145@tcp:38:kworker.0
00000100:00080000:2.0:1787827956.822859:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1787827956.822880:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787827956.822886:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827956.822894:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827956/real 1787827956]  req@ffff8cd0c3da83c0 x1874671626791168/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827981 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1787827956.822899:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827956.822921:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626791424/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827956.822926:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626791168/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827981 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827956.822948:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827956.822965:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827956.822979:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626791168:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827956.822989:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626791296:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827956.823009:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827956.823021:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827956/real 1787827956]  req@ffff8cd0c3da9680 x1874671626791296/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827981 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827956.823030:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626791296/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827981 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827956.823035:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827956.823037:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827956.823039:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626791296:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827956.823050:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626791424:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827956.823070:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827956.823082:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827956/real 1787827956]  req@ffff8cd0c3da8f00 x1874671626791424/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827981 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827956.823119:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626791424/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827981 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827956.823144:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827956.823147:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827956.823150:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626791424:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827957.014231:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827957.398321:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827958.038187:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827958.422335:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827959.062143:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827959.446236:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827960.086176:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827960.470336:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827961.110215:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827961.494268:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827961.942226:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827961.942239:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827961.942245:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827961.942258:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1847
00000100:00080000:2.0:1787827961.942273:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827961.942284:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827961.942337:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626791552/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827961.942372:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827961.942377:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827961.942380:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827961.942384:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1847
00000100:00080000:2.0:1787827961.942388:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827961.942393:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827961.942406:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827961.942443:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626791680/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827961.942461:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787827961.942466:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626791552:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787827961.942467:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827961.942474:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827961.942487:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1847
00000100:00080000:2.0:1787827961.942492:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827961.942497:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827961.942506:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827961.942524:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626791808/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827961.942537:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827961.942541:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827961.942544:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827961.942550:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1847
00000100:00080000:2.0:1787827961.942554:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827961.942559:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827961.942565:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787827961.942568:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827961.942587:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab480 x1874671626791936/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827961.942632:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827961/real 1787827961]  req@ffff8cd0c3da9a40 x1874671626791552/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827986 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827961.942656:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626791552/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827986 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827961.942669:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827961.942675:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827961.942680:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626791552:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827961.942709:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626791680:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827961.942735:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827961.942750:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827961/real 1787827961]  req@ffff8cd0c3dab0c0 x1874671626791680/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827986 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827961.942842:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626791680/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827986 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827961.942852:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827961.942872:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827961.942877:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626791680:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827961.942886:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626791808:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827961.942917:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827961.942940:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827961/real 1787827961]  req@ffff8cd0c3daa940 x1874671626791808/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827986 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827961.942956:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626791808/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827986 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827961.942966:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827961.942969:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827961.942984:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626791808:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827961.942998:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626791936:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827961.943019:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827961.943032:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827961/real 1787827961]  req@ffff8cd0c3dab480 x1874671626791936/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827986 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827961.943046:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab480 x1874671626791936/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827986 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827961.943086:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827961.943090:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827961.943094:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626791936:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827962.134224:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827962.518325:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827963.158269:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827963.542244:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827964.182195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827964.566362:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827965.206212:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827965.590353:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827966.230214:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827966.614329:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827967.062281:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827967.062297:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827967.062337:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827967.062354:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1852
00000100:00080000:2.0:1787827967.062376:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827967.062390:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827967.062463:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626792064/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827967.062517:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827967.062522:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827967.062524:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827967.062530:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1852
00000100:00080000:2.0:1787827967.062536:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827967.062541:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827967.062556:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827967.062576:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626792192/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827967.062590:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827967.062595:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827967.062597:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827967.062602:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1852
00000100:00080000:2.0:1787827967.062607:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827967.062611:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827967.062619:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827967.062635:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626792320/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827967.062676:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626792064:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787827967.062681:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827967.062704:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827967.062709:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827967.062743:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1852
00000100:00080000:2.0:1787827967.062752:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827967.062760:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827967.062771:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827967.062798:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626792448/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787827967.062851:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827967.062930:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827967/real 1787827967]  req@ffff8cd0c3da8780 x1874671626792064/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827992 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827967.062972:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626792064/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827992 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827967.062994:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827967.063003:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827967.063012:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626792064:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827967.063031:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626792192:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827967.063081:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827967.063152:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827967/real 1787827967]  req@ffff8cd0c3dabc00 x1874671626792192/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827992 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827967.063202:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626792192/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827992 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827967.063220:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827967.063228:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827967.063235:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626792192:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827967.063277:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626792320:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827967.063323:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827967.063350:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827967/real 1787827967]  req@ffff8cd0c3da8000 x1874671626792320/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827992 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827967.063376:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626792320/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827992 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827967.063393:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827967.063399:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827967.063407:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626792320:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827967.063421:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626792448:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827967.063462:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827967.063484:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827967/real 1787827967]  req@ffff8cd0c3dab840 x1874671626792448/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827992 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827967.063510:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626792448/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827992 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827967.063550:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827967.063555:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827967.063560:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626792448:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827967.254169:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827967.638303:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827968.278170:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827968.662331:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827969.302214:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827969.686338:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827970.326180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827970.710283:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827971.350172:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827971.734330:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827972.182219:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827972.182232:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827972.182238:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827972.182251:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1858
00000100:00080000:2.0:1787827972.182265:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827972.182275:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827972.182326:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626792576/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827972.182370:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827972.182375:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827972.182377:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827972.182403:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1858
00000100:00080000:2.0:1787827972.182410:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827972.182417:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827972.182432:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827972.182451:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626792704/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827972.182469:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787827972.182471:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626792576:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787827972.182477:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827972.182482:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827972.182494:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1858
00000100:00080000:2.0:1787827972.182498:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827972.182502:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827972.182508:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827972.182525:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626792832/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827972.182539:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827972.182544:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827972.182547:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827972.182553:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1858
00000100:00080000:2.0:1787827972.182571:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827972.182576:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827972.182581:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787827972.182594:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827972.182597:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626792960/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827972.182672:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827972/real 1787827972]  req@ffff8cd0c3daa580 x1874671626792576/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827997 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827972.182733:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626792576/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827997 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827972.182755:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827972.182763:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827972.182772:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626792576:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827972.182809:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626792704:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827972.182847:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827972.182866:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827972/real 1787827972]  req@ffff8cd0c3da9e00 x1874671626792704/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827997 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827972.182908:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626792704/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827997 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827972.182919:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827972.182923:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827972.182928:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626792704:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827972.182936:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626792832:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827972.182964:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827972.182977:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827972/real 1787827972]  req@ffff8cd0c3da92c0 x1874671626792832/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827997 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827972.182992:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626792832/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827997 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827972.183002:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827972.183006:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827972.183010:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626792832:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827972.183016:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626792960:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827972.183035:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827972.183046:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827972/real 1787827972]  req@ffff8cd0c3daa1c0 x1874671626792960/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827997 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827972.183060:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626792960/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827997 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827972.183127:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827972.183132:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827972.183137:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626792960:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827972.374164:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827972.758359:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827973.398214:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827973.782229:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827974.422178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827974.806245:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827975.446167:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827975.830214:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827976.470176:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827976.854233:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827977.302222:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827977.302231:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827977.302235:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827977.302243:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1863
00000100:00080000:2.0:1787827977.302254:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827977.302261:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827977.302296:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daad00 x1874671626793088/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827977.302321:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827977.302324:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827977.302325:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827977.302328:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1863
00000100:00080000:2.0:1787827977.302331:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827977.302333:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827977.302341:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827977.302350:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626793216/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827977.302357:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827977.302359:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827977.302361:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827977.302363:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1863
00000100:00080000:2.0:1787827977.302380:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827977.302384:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827977.302389:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827977.302397:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626793344/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827977.302406:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626793088:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787827977.302408:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827977.302415:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827977.302417:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827977.302421:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1863
00000100:00080000:2.0:1787827977.302423:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827977.302425:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827977.302429:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827977.302436:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626793472/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787827977.302469:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827977.302506:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827977/real 1787827977]  req@ffff8cd0c3daad00 x1874671626793088/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828002 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827977.302519:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daad00 x1874671626793088/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828002 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827977.302528:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827977.302530:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827977.302533:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626793088:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827977.302538:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626793216:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827977.302552:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827977.302560:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827977/real 1787827977]  req@ffff8cd0c3daa1c0 x1874671626793216/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828002 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827977.302576:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626793216/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828002 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827977.302582:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827977.302584:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827977.302586:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626793216:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827977.302603:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626793344:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787827977.302617:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827977.302624:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827977/real 1787827977]  req@ffff8cd0c3da92c0 x1874671626793344/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828002 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827977.302632:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626793344/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828002 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827977.302637:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827977.302639:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827977.302640:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626793344:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827977.302644:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626793472:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787827977.302699:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827977.302716:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827977/real 1787827977]  req@ffff8cd0c3da9e00 x1874671626793472/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828002 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827977.302731:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626793472/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828002 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827977.302759:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827977.302763:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827977.302768:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626793472:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827977.494205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827977.878347:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827978.518261:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827978.902266:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827979.542217:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827979.926306:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827980.566204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827980.950318:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827981.590197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827981.974312:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787827982.422230:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827982.422247:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787827982.422254:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827982.422270:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1868
00000100:00080000:2.0:1787827982.422288:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827982.422301:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827982.422378:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626793600/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827982.422447:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827982.422455:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827982.422460:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827982.422470:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1868
00000100:00080000:2.0:1787827982.422481:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827982.422492:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827982.422512:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827982.422513:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626793600:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827982.422548:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626793728/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827982.422575:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787827982.422583:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787827982.422589:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827982.422597:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1868
00000100:00080000:2.0:1787827982.422606:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827982.422650:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827982.422663:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827982.422677:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626793728:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827982.422695:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626793856/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787827982.422725:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787827982.422735:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626793856:192.168.203.145@tcp:8:kworker.0
00000100:00080000:2.0:1787827982.422737:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787827982.422748:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787827982.422764:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1868
00000100:00080000:2.0:1787827982.422789:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787827982.422801:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787827982.422814:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787827982.422841:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626793984/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787827982.422935:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626793984:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827982.614233:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827982.998334:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827983.638231:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827984.022328:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827984.662215:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827985.046302:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:1.0:1787827985.494493:0:2344:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:1.0:1787827985.494563:0:2344:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1787827985.494612:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827985.494675:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827985.494693:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787827985.494718:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:3.0:1787827985.494778:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827982/real 1787827985]  req@ffff8cd0c3dabc00 x1874671626793984/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828007 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827985.494830:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626793984/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828007 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827985.494890:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827985.494900:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827985.494909:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626793984:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827985.494924:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827982/real 1787827985]  req@ffff8cd0c3da8000 x1874671626793856/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828007 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827985.494952:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626793856/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828007 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827985.494968:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827985.494975:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827985.494981:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626793856:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787827985.495003:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827982/real 1787827985]  req@ffff8cd0c3dab840 x1874671626793728/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828007 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827985.495028:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626793728/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828007 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827985.495065:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827985.495073:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827985.495080:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626793728:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827985.495149:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827982/real 1787827985]  req@ffff8cd0c3daa580 x1874671626793600/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828007 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827985.495177:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626793600/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828007 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827985.495209:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787827985.495218:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787827985.495225:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626793600:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787827985.686209:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827986.070244:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787827986.710210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827987.094250:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827987.542189:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827987.542198:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827987.542202:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827987.542228:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1873
00000100:00080000:3.0:1787827987.542238:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827987.542245:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827987.542275:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626794112/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827987.542336:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827987.542341:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827987.542343:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827987.542363:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1873
00000100:00080000:3.0:1787827987.542366:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827987.542369:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827987.542377:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827987.542387:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626794240/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827987.542407:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626794112:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827987.542415:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827987.542419:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827987.542421:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827987.542424:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1873
00000100:00080000:3.0:1787827987.542444:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827987.542449:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827987.542456:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827987.542471:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626794368/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827987.542483:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827987.542488:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827987.542490:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827987.542496:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1873
00000100:00080000:3.0:1787827987.542505:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827987.542512:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827987.542537:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787827987.542547:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827987.542551:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626794496/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827987.542607:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827987/real 1787827987]  req@ffff8cd0c8eada40 x1874671626794112/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828012 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827987.542632:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626794112/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828012 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827987.542643:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827987.542647:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827987.542667:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626794112:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827987.542691:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626794240:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827987.542732:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827987.542746:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827987/real 1787827987]  req@ffff8cd0c8eaf0c0 x1874671626794240/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828012 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827987.542758:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626794240/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828012 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827987.542781:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827987.542785:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827987.542788:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626794240:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827987.542794:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626794368:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827987.542818:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827987.542831:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827987/real 1787827987]  req@ffff8cd0c8eac3c0 x1874671626794368/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828012 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827987.542842:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626794368/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828012 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827987.542857:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827987.542860:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827987.542863:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626794368:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827987.542869:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626794496:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827987.542889:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827987.542901:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827987/real 1787827987]  req@ffff8cd0c8ead2c0 x1874671626794496/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828012 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827987.542913:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626794496/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828012 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827987.542921:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827987.542924:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827987.542937:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626794496:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827987.734198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827988.118321:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827988.758197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827989.142286:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827989.782193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827990.166349:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827990.806224:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827991.190332:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827991.830208:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827992.214295:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827992.662161:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827992.662170:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827992.662190:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827992.662199:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1878
00000100:00080000:3.0:1787827992.662209:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827992.662216:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827992.662263:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626794624/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827992.662288:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827992.662291:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827992.662293:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827992.662296:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1878
00000100:00080000:3.0:1787827992.662299:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827992.662302:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827992.662311:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827992.662320:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626794752/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827992.662329:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827992.662332:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827992.662334:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827992.662337:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1878
00000100:00080000:3.0:1787827992.662339:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827992.662344:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787827992.662346:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626794624:192.168.203.145@tcp:250:kworker.0
00000008:00100000:3.0:1787827992.662348:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827992.662359:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626794880/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827992.662379:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827992.662383:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827992.662384:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827992.662389:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1878
00000100:00080000:3.0:1787827992.662401:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827992.662404:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827992.662409:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787827992.662411:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787827992.662423:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626795008/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827992.662455:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827992/real 1787827992]  req@ffff8cd0c8eac780 x1874671626794624/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828017 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827992.662472:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626794624/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828017 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827992.662481:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827992.662485:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827992.662488:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626794624:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787827992.662494:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626794752:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827992.662511:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827992.662521:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827992/real 1787827992]  req@ffff8cd0c8eaf840 x1874671626794752/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828017 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827992.662530:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626794752/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828017 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827992.662566:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827992.662569:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827992.662572:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626794752:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827992.662596:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626794880:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827992.662616:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827992.662625:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827992/real 1787827992]  req@ffff8cd0c8eae1c0 x1874671626794880/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828017 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827992.662635:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626794880/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828017 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827992.662642:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827992.662645:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827992.662647:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626794880:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827992.662651:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626795008:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827992.662663:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827992.662669:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827992/real 1787827992]  req@ffff8cd0c8eae940 x1874671626795008/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828017 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827992.662678:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626795008/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828017 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827992.662684:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827992.662686:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827992.662695:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626795008:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827992.854166:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827993.238231:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827993.878205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827994.262338:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827994.902236:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827995.286295:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827995.926221:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827996.310305:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827996.950280:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827997.334334:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787827997.782213:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827997.782240:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787827997.782247:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827997.782262:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1883
00000100:00080000:3.0:1787827997.782279:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827997.782290:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787827997.782350:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626795136/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827997.782390:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827997.782394:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827997.782397:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827997.782424:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1883
00000100:00080000:3.0:1787827997.782430:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827997.782437:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827997.782452:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787827997.782473:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626795264/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827997.782488:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626795136:192.168.203.145@tcp:250:kworker.0
00000800:00000100:2.0:1787827997.782597:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827997.782645:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827997/real 1787827997]  req@ffff8cd0c8eacf00 x1874671626795136/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828022 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827997.782663:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626795136/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828022 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827997.782675:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827997.782680:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827997.782684:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626795136:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787827997.782770:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787827997.782776:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787827997.782779:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827997.782787:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1883
00000100:00080000:3.0:1787827997.782796:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1787827997.782800:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626795264:192.168.203.145@tcp:38:kworker.0
00000100:00080000:3.0:1787827997.782811:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787827997.782829:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787827997.782843:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827997.782875:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827997/real 1787827997]  req@ffff8cd0c8eac000 x1874671626795264/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828022 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787827997.782877:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626795392/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787827997.782946:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1787827997.782950:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626795264/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828022 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787827997.782952:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787827997.782958:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787827997.782970:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1883
00000100:00080000:3.0:1787827997.782978:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787827997.782993:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00080000:2.0:1787827997.782994:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000008:00100000:3.0:1787827997.783005:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1787827997.783022:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827997.783027:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626795264:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787827997.783042:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626795520/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787827997.783053:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626795392:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787827997.783193:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827997.783208:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827997/real 1787827997]  req@ffff8cd0c8eacf00 x1874671626795392/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828022 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827997.783221:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626795392/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828022 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827997.783231:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827997.783236:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827997.783241:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626795392:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787827997.783261:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626795520:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787827997.783293:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787827997.783309:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827997/real 1787827997]  req@ffff8cd0c8eae940 x1874671626795520/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828022 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787827997.783321:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626795520/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828022 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787827997.783329:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787827997.783332:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787827997.783335:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626795520:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787827997.974207:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787827998.358364:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787827998.998185:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787827999.382236:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828000.022185:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828000.406206:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828001.046247:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828001.430259:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828002.070172:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828002.454351:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828002.902184:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828002.902192:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828002.902196:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828002.902204:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1888
00000100:00080000:1.0:1787828002.902229:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828002.902236:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828002.902266:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626795648/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828002.902289:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828002.902292:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828002.902293:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828002.902296:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1888
00000100:00080000:1.0:1787828002.902298:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828002.902300:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828002.902309:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828002.902317:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626795776/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828002.902325:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828002.902328:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787828002.902330:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828002.902332:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1888
00000100:00080000:1.0:1787828002.902334:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1787828002.902338:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626795648:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787828002.902339:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828002.902345:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828002.902353:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626795904/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828002.902359:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828002.902367:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828002.902368:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828002.902371:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1888
00000100:00080000:1.0:1787828002.902375:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828002.902377:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828002.902382:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828002.902390:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626796032/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787828002.902403:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828002.902452:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828002/real 1787828002]  req@ffff8cd0d48f70c0 x1874671626795648/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828027 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828002.902480:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626795648/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828027 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828002.902489:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828002.902492:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828002.902495:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626795648:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828002.902514:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626795776:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828002.902532:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828002.902541:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828002/real 1787828002]  req@ffff8cd0d48f43c0 x1874671626795776/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828027 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828002.902550:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626795776/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828027 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828002.902556:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828002.902558:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828002.902575:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626795776:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828002.902580:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626795904:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828002.902596:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828002.902604:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828002/real 1787828002]  req@ffff8cd0d48f5a40 x1874671626795904/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828027 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828002.902612:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626795904/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828027 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828002.902617:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828002.902619:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828002.902621:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626795904:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828002.902625:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626796032:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828002.902635:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828002.902641:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828002/real 1787828002]  req@ffff8cd0d48f7840 x1874671626796032/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828027 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828002.902648:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626796032/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828027 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828002.902653:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828002.902655:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828002.902657:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626796032:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828003.094185:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828003.478195:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828004.118185:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828004.502270:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828005.142222:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828005.526326:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828006.166224:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828006.550391:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828007.190264:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828007.574352:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828008.022253:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828008.022271:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828008.022278:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828008.022295:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1893
00000100:00080000:2.0:1787828008.022314:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828008.022328:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828008.022390:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626796160/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828008.022458:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828008.022467:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828008.022471:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828008.022480:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1893
00000100:00080000:2.0:1787828008.022490:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828008.022500:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828008.022518:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626796160:192.168.203.145@tcp:250:kworker.0
00000008:00100000:2.0:1787828008.022519:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828008.022561:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab480 x1874671626796288/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828008.022588:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828008.022595:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828008.022600:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:3.0:1787828008.022620:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787828008.022640:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1893
00000100:00080000:2.0:1787828008.022652:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828008.022661:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828008.022674:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828008.022687:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828008/real 1787828008]  req@ffff8cd0c3da8780 x1874671626796160/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828033 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828008.022708:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626796416/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828008.022722:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626796160/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828033 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828008.022747:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828008.022754:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828008.022757:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828008.022766:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828008.022767:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828008.022783:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626796160:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828008.022787:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1893
00000100:00080000:2.0:1787828008.022802:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828008.022812:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828008.022823:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828008.022827:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626796288:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828008.022858:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626796544/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828008.022874:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828008.022895:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828008/real 1787828008]  req@ffff8cd0c3dab480 x1874671626796288/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828033 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828008.022925:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab480 x1874671626796288/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828033 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828008.022938:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828008.022944:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828008.022963:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626796288:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828008.022973:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626796416:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828008.023003:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828008.023017:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828008/real 1787828008]  req@ffff8cd0c3daa940 x1874671626796416/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828033 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828008.023033:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626796416/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828033 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828008.023043:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828008.023048:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828008.023052:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626796416:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828008.023070:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626796544:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828008.023153:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828008.023169:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828008/real 1787828008]  req@ffff8cd0c3dab0c0 x1874671626796544/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828033 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828008.023185:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626796544/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828033 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828008.023196:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828008.023200:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828008.023204:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626796544:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828008.214199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828008.598357:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828009.238289:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828009.622209:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828010.262169:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828010.646352:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828011.286269:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828011.670294:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828012.310146:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828012.694295:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828013.142215:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828013.142229:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828013.142258:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828013.142271:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1898
00000100:00080000:2.0:1787828013.142286:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828013.142295:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828013.142343:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626796672/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828013.142383:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828013.142387:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828013.142390:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828013.142395:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1898
00000100:00080000:2.0:1787828013.142424:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828013.142432:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828013.142446:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828013.142465:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626796800/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828013.142478:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626796672:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828013.142487:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828013.142494:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828013.142497:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828013.142503:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1898
00000100:00080000:2.0:1787828013.142508:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828013.142512:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828013.142519:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828013.142538:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626796928/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828013.142570:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:3.0:1787828013.142579:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787828013.142581:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828013.142590:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828013.142598:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1898
00000100:00080000:2.0:1787828013.142607:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828013.142614:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828013.142621:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828013.142643:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626797056/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828013.142665:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828013/real 1787828013]  req@ffff8cd0c3da9a40 x1874671626796672/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828038 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828013.142696:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626796672/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828038 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828013.142713:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828013.142720:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828013.142728:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626796672:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828013.142761:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626796800:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828013.142792:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828013.142808:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828013/real 1787828013]  req@ffff8cd0c3da8f00 x1874671626796800/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828038 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828013.142837:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626796800/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828038 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828013.142849:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828013.142853:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828013.142858:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626796800:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828013.142865:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626796928:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828013.142889:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828013.142907:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828013/real 1787828013]  req@ffff8cd0c3da9680 x1874671626796928/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828038 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828013.142945:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626796928/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828038 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828013.142956:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828013.142960:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828013.142964:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626796928:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828013.142971:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626797056:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828013.142997:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828013.143010:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828013/real 1787828013]  req@ffff8cd0c3da83c0 x1874671626797056/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828038 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828013.143024:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626797056/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828038 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828013.143034:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828013.143037:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828013.143041:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626797056:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828013.334262:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828013.718288:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828014.358261:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828014.742266:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828015.382241:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828015.766211:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828016.406244:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828016.790301:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828017.430228:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828017.814320:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828018.262154:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828018.262165:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828018.262170:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828018.262181:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1904
00000100:00080000:2.0:1787828018.262199:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828018.262206:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828018.262255:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8b40 x1874671626797184/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828018.262281:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828018.262296:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828018.262298:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828018.262301:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1904
00000100:00080000:2.0:1787828018.262305:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828018.262309:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828018.262319:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828018.262333:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626797312/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828018.262340:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626797184:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828018.262344:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828018.262351:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828018.262353:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828018.262375:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1904
00000100:00080000:2.0:1787828018.262379:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828018.262382:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828018.262387:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828018.262397:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626797440/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828018.262404:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828018.262407:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828018.262409:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828018.262412:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1904
00000100:00080000:2.0:1787828018.262419:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828018.262422:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828018.262431:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828018.262441:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626797568/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828018.262474:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828018.262524:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828018/real 1787828018]  req@ffff8cd0c3da8b40 x1874671626797184/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828043 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828018.262555:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8b40 x1874671626797184/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828043 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828018.262564:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828018.262568:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828018.262571:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626797184:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828018.262593:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626797312:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828018.262612:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828018.262622:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828018/real 1787828018]  req@ffff8cd0c3da83c0 x1874671626797312/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828043 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828018.262630:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626797312/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828043 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828018.262635:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828018.262637:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828018.262639:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626797312:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828018.262643:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626797440:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828018.262655:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828018.262661:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828018/real 1787828018]  req@ffff8cd0c3da9680 x1874671626797440/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828043 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828018.262668:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626797440/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828043 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828018.262682:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828018.262685:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828018.262687:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626797440:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828018.262691:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626797568:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828018.262707:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828018.262717:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828018/real 1787828018]  req@ffff8cd0c3da8f00 x1874671626797568/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828043 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828018.262731:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626797568/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828043 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828018.262739:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828018.262742:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828018.262746:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626797568:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828018.454192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828018.838245:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828019.478304:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828019.862260:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828020.502294:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828020.886396:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828021.526259:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828021.910441:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828022.550282:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828022.934355:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828023.382241:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828023.382253:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828023.382258:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828023.382271:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1909
00000100:00080000:1.0:1787828023.382285:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828023.382292:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828023.382400:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626797696/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828023.382439:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828023.382444:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828023.382446:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828023.382450:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1909
00000100:00080000:1.0:1787828023.382455:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828023.382460:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828023.382485:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828023.382504:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626797824/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828023.382535:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828023.382539:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787828023.382542:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828023.382564:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1909
00000100:00100000:3.0:1787828023.382567:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626797696:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787828023.382572:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828023.382583:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828023.382592:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828023.382611:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626797952/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828023.382623:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828023.382627:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828023.382630:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828023.382642:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1909
00000100:00080000:1.0:1787828023.382648:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828023.382654:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828023.382660:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787828023.382670:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787828023.382677:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626798080/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828023.382758:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828023/real 1787828023]  req@ffff8cd0d48f4b40 x1874671626797696/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828048 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828023.382802:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626797696/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828048 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828023.382815:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828023.382837:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828023.382844:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626797696:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828023.382882:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626797824:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828023.382927:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828023.382946:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828023/real 1787828023]  req@ffff8cd0d48f6580 x1874671626797824/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828048 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828023.382960:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626797824/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828048 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828023.382970:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828023.382974:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828023.382978:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626797824:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828023.382985:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626797952:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828023.383019:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828023.383049:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828023/real 1787828023]  req@ffff8cd0d48f7480 x1874671626797952/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828048 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828023.383061:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626797952/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828048 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828023.383070:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828023.383073:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828023.383076:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626797952:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828023.383117:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626798080:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828023.383153:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828023.383166:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828023/real 1787828023]  req@ffff8cd0d48f4000 x1874671626798080/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828048 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828023.383179:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626798080/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828048 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828023.383187:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828023.383191:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828023.383194:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626798080:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828023.574264:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828023.959169:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828024.598284:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828024.982301:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828025.622229:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828026.006223:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828026.646242:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828027.030350:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828027.670191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828028.054316:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828028.502335:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828028.502352:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828028.502359:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828028.502376:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1914
00000100:00080000:2.0:1787828028.502393:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828028.502407:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828028.502459:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626798208/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828028.502501:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828028.502506:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828028.502538:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828028.502544:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1914
00000100:00080000:2.0:1787828028.502550:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828028.502555:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828028.502569:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828028.502586:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626798336/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828028.502599:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626798208:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828028.502608:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828028.502614:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828028.502617:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828028.502623:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1914
00000100:00080000:2.0:1787828028.502628:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828028.502632:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828028.502638:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828028.502653:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626798464/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828028.502666:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828028.502671:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828028.502673:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828028.502677:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1914
00000100:00080000:2.0:1787828028.502682:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1787828028.502689:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787828028.502695:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828028.502703:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828028.502726:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab480 x1874671626798592/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828028.502761:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828028/real 1787828028]  req@ffff8cd0c3da9a40 x1874671626798208/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828053 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828028.502790:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626798208/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828053 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828028.502804:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828028.502809:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828028.502814:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626798208:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828028.502841:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626798336:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828028.502868:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828028.502897:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828028/real 1787828028]  req@ffff8cd0c3dab0c0 x1874671626798336/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828053 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828028.502912:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626798336/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828053 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828028.502923:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828028.502927:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828028.502931:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626798336:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828028.502938:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626798464:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828028.502961:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828028.502973:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828028/real 1787828028]  req@ffff8cd0c3daa940 x1874671626798464/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828053 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828028.502986:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626798464/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828053 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828028.502995:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828028.502999:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828028.503002:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626798464:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828028.503008:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626798592:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828028.503027:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828028.503045:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828028/real 1787828028]  req@ffff8cd0c3dab480 x1874671626798592/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828053 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828028.503068:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab480 x1874671626798592/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828053 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828028.503134:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828028.503139:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828028.503143:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626798592:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828028.694215:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828029.078375:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828029.718203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828030.102327:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828030.742227:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828031.126328:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828031.766260:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828032.150340:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828032.790187:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828033.174247:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828033.622183:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828033.622190:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828033.622194:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828033.622202:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1919
00000100:00080000:1.0:1787828033.622228:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828033.622237:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828033.622286:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626798720/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828033.622321:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828033.622323:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828033.622325:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828033.622328:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1919
00000100:00080000:1.0:1787828033.622332:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828033.622335:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828033.622345:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828033.622355:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626798848/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828033.622365:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787828033.622366:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626798720:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787828033.622371:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787828033.622375:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828033.622379:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1919
00000100:00080000:1.0:1787828033.622381:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828033.622383:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828033.622386:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828033.622395:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626798976/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828033.622402:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828033.622410:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828033.622412:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828033.622415:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1919
00000100:00080000:1.0:1787828033.622426:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828033.622432:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:3.0:1787828033.622432:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:1.0:1787828033.622439:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828033.622449:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626799104/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828033.622481:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828033/real 1787828033]  req@ffff8cd0d48f4000 x1874671626798720/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828058 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828033.622495:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626798720/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828058 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828033.622504:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828033.622507:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828033.622510:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626798720:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828033.622529:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626798848:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828033.622546:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828033.622555:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828033/real 1787828033]  req@ffff8cd0d48f7480 x1874671626798848/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828058 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828033.622571:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626798848/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828058 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828033.622577:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828033.622585:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828033.622588:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626798848:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828033.622592:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626798976:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828033.622609:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828033.622616:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828033/real 1787828033]  req@ffff8cd0d48f6580 x1874671626798976/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828058 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828033.622624:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626798976/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828058 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828033.622629:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828033.622631:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828033.622633:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626798976:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828033.622636:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626799104:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828033.622647:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828033.622653:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828033/real 1787828033]  req@ffff8cd0d48f4b40 x1874671626799104/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828058 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828033.622660:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626799104/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828058 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828033.622665:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828033.622667:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828033.622669:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626799104:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828033.814131:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828034.198333:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828034.838159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828035.222329:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828035.862168:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828036.246344:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828036.886220:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828037.270356:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828037.910335:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828038.294329:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828038.742281:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828038.742294:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828038.742299:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828038.742312:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1924
00000100:00080000:2.0:1787828038.742324:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828038.742332:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828038.742374:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626799232/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828038.742404:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828038.742407:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828038.742410:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828038.742431:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1924
00000100:00080000:2.0:1787828038.742436:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828038.742440:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828038.742451:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828038.742459:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626799232:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828038.742468:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626799360/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828038.742481:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828038.742484:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828038.742486:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828038.742490:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1924
00000100:00080000:2.0:1787828038.742493:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828038.742497:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828038.742502:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828038.742512:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626799488/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828038.742521:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828038.742527:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000800:00000100:3.0:1787828038.742527:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787828038.742530:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828038.742536:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1924
00000100:00080000:2.0:1787828038.742540:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828038.742543:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828038.742548:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828038.742559:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626799616/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828038.742584:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828038/real 1787828038]  req@ffff8cd0c3da8780 x1874671626799232/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828063 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828038.742607:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626799232/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828063 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828038.742619:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828038.742624:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828038.742628:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626799232:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828038.742651:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626799360:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828038.742672:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828038.742683:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828038/real 1787828038]  req@ffff8cd0c3daa580 x1874671626799360/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828063 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828038.742695:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626799360/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828063 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828038.742703:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828038.742705:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828038.742708:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626799360:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828038.742714:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626799488:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828038.742729:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828038.742750:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828038/real 1787828038]  req@ffff8cd0c3dab840 x1874671626799488/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828063 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828038.742761:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626799488/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828063 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828038.742769:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828038.742772:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828038.742774:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626799488:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828038.742779:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626799616:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828038.742795:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828038.742803:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828038/real 1787828038]  req@ffff8cd0c3da8000 x1874671626799616/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828063 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828038.742814:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626799616/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828063 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828038.742820:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828038.742823:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828038.742825:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626799616:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828038.934173:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828039.318257:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828039.958279:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828040.342331:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828040.982234:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828041.366389:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828042.006262:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828042.390299:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828043.030214:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828043.414255:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828043.862256:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828043.862270:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828043.862287:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828043.862299:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1929
00000100:00080000:2.0:1787828043.862313:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828043.862323:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828043.862373:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626799744/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828043.862413:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828043.862418:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828043.862421:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828043.862426:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1929
00000100:00080000:2.0:1787828043.862430:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828043.862435:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828043.862449:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828043.862465:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626799872/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828043.862497:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626799744:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828043.862532:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828043.862538:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828043.862541:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828043.862549:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1929
00000100:00080000:2.0:1787828043.862556:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828043.862563:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828043.862574:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828043.862591:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626800000/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828043.862605:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828043.862611:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828043.862614:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828043.862619:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1929
00000100:00080000:2.0:1787828043.862623:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828043.862634:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:3.0:1787828043.862635:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:2.0:1787828043.862650:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828043.862669:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626800128/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828043.862716:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828043/real 1787828043]  req@ffff8cd0c3dabc00 x1874671626799744/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828068 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828043.862765:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626799744/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828068 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828043.862778:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828043.862783:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828043.862787:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626799744:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828043.862821:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626799872:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828043.862863:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828043.862876:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828043/real 1787828043]  req@ffff8cd0c3da9e00 x1874671626799872/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828068 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828043.862885:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626799872/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828068 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828043.862892:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828043.862895:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828043.862897:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626799872:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828043.862902:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626800000:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828043.862918:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828043.862928:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828043/real 1787828043]  req@ffff8cd0c3da92c0 x1874671626800000/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828068 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828043.862943:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626800000/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828068 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828043.862968:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828043.862973:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828043.862977:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626800000:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828043.862986:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626800128:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828043.863009:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828043.863020:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828043/real 1787828043]  req@ffff8cd0c3daa1c0 x1874671626800128/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828068 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828043.863029:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626800128/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828068 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828043.863036:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828043.863038:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828043.863042:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626800128:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828044.054262:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828044.438254:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828045.078492:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828045.462352:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828046.102233:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828046.486328:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828047.126255:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828047.510266:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828048.150227:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828048.534280:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828048.982320:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828048.982334:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828048.982340:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828048.982353:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1934
00000100:00080000:0.0:1787828048.982367:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828048.982377:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828048.982439:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626800256/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828048.982475:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828048.982480:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828048.982483:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828048.982488:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1934
00000100:00080000:0.0:1787828048.982493:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828048.982497:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828048.982510:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828048.982527:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626800384/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828048.982541:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828048.982546:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828048.982549:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828048.982563:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1934
00000100:00080000:0.0:1787828048.982590:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1787828048.982597:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626800256:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828048.982603:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828048.982618:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828048.982640:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626800512/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828048.982657:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828048.982671:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828048.982675:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828048.982684:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1934
00000100:00080000:0.0:1787828048.982692:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828048.982701:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828048.982711:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828048.982739:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626800640/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828048.982757:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626800384:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828048.982825:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626800512:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828048.982855:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626800640:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828049.174174:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828049.558296:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828050.198251:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828050.582308:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828051.222261:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828051.606317:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:1.0:1787828052.054469:0:2346:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:1.0:1787828052.054543:0:2346:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1787828052.054564:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787828052.054617:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787828052.054627:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787828052.054637:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:3.0:1787828052.054749:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828048/real 1787828052]  req@ffff8cd0c8e9de00 x1874671626800512/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828073 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828052.054798:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626800512/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828073 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828052.054819:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828052.054828:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828052.054835:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626800512:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828052.054849:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828048/real 1787828052]  req@ffff8cd0c8e9e580 x1874671626800640/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828073 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828052.054871:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626800640/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828073 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828052.054886:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828052.054893:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828052.054900:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626800640:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828052.054912:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828048/real 1787828052]  req@ffff8cd0c8e9fc00 x1874671626800256/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828073 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828052.054933:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626800256/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828073 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828052.054949:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828052.054955:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828052.054962:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626800256:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828052.054974:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828048/real 1787828052]  req@ffff8cd0c8e9cf00 x1874671626800384/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828073 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828052.055032:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626800384/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828073 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828052.055048:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828052.055054:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828052.055060:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626800384:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828052.246291:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828052.630280:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828053.270226:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828053.654337:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828054.102340:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828054.102354:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828054.102359:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828054.102372:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1939
00000100:00080000:1.0:1787828054.102385:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828054.102395:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828054.102442:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626800768/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828054.102503:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828054.102511:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828054.102535:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828054.102544:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1939
00000100:00080000:1.0:1787828054.102554:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828054.102562:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828054.102580:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828054.102611:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626800896/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828054.102636:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626800768:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787828054.102640:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828054.102656:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787828054.102659:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828054.102666:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1939
00000100:00080000:1.0:1787828054.102672:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828054.102679:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828054.102688:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828054.102709:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626801024/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828054.102723:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828054.102728:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828054.102731:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828054.102737:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1939
00000100:00080000:1.0:1787828054.102742:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828054.102747:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828054.102755:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787828054.102757:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787828054.102796:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626801152/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828054.102835:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828054/real 1787828054]  req@ffff8cd0d48f7840 x1874671626800768/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828079 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828054.102878:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626800768/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828079 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828054.102900:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828054.102908:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828054.102916:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626800768:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828054.102956:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626800896:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828054.103007:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828054.103035:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828054/real 1787828054]  req@ffff8cd0d48f5a40 x1874671626800896/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828079 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828054.103062:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626800896/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828079 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828054.103187:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828054.103195:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828054.103202:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626800896:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828054.103215:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626801024:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828054.103260:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828054.103284:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828054/real 1787828054]  req@ffff8cd0d48f43c0 x1874671626801024/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828079 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828054.103319:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626801024/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828079 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828054.103336:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828054.103343:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828054.103350:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626801024:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828054.103362:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626801152:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828054.104200:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828054.104875:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828054/real 1787828054]  req@ffff8cd0d48f70c0 x1874671626801152/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828079 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828054.104906:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626801152/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828079 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828054.104924:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828054.104932:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828054.104956:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626801152:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828054.294235:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828054.678319:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828055.318298:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828055.702298:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828056.342266:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828056.726291:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828057.366270:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828057.750379:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828058.390262:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828058.774277:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828059.222205:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828059.222215:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828059.222237:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828059.222250:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1945
00000100:00080000:1.0:1787828059.222264:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828059.222274:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828059.222333:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626801280/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828059.222364:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828059.222368:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828059.222370:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828059.222374:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1945
00000100:00080000:1.0:1787828059.222378:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828059.222381:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828059.222416:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828059.222431:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626801408/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828059.222444:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828059.222449:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:3.0:1787828059.222449:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626801280:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787828059.222454:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828059.222465:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1945
00000100:00080000:1.0:1787828059.222469:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828059.222505:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828059.222511:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828059.222526:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626801536/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828059.222537:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828059.222541:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828059.222543:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828059.222546:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1945
00000100:00080000:1.0:1787828059.222552:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828059.222560:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:3.0:1787828059.222560:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:1.0:1787828059.222570:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828059.222581:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626801664/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828059.222632:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828059/real 1787828059]  req@ffff8cd0d48f61c0 x1874671626801280/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828084 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828059.222654:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626801280/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828084 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828059.222666:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828059.222671:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828059.222675:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626801280:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828059.222718:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626801408:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828059.222746:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828059.222759:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828059/real 1787828059]  req@ffff8cd0d48f7c00 x1874671626801408/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828084 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828059.222771:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626801408/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828084 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828059.222779:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828059.222783:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828059.222786:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626801408:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828059.222804:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626801536:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828059.222826:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828059.222837:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828059/real 1787828059]  req@ffff8cd0d48f52c0 x1874671626801536/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828084 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828059.222848:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626801536/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828084 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828059.222855:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828059.222858:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828059.222861:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626801536:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828059.222866:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626801664:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828059.222880:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828059.222888:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828059/real 1787828059]  req@ffff8cd0d48f6d00 x1874671626801664/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828084 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828059.222899:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626801664/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828084 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828059.222922:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828059.222927:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828059.222931:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626801664:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828059.414228:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828059.798365:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828060.438169:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828060.822299:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828061.462251:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828061.846246:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828062.486203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828062.870384:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828063.510203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828063.894304:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828064.342210:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828064.342225:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828064.342232:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828064.342249:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1950
00000100:00080000:1.0:1787828064.342267:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828064.342280:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828064.342352:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626801792/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828064.342428:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828064.342436:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828064.342459:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828064.342469:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1950
00000100:00080000:1.0:1787828064.342479:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828064.342488:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828064.342495:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626801792:192.168.203.145@tcp:250:kworker.0
00000008:00100000:1.0:1787828064.342509:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828064.342567:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626801920/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828064.342586:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828064.342592:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787828064.342595:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828064.342600:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1950
00000100:00080000:1.0:1787828064.342606:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1787828064.342608:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787828064.342616:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828064.342627:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828064.342645:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626802048/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828064.342657:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828064.342662:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:3.0:1787828064.342667:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828064/real 1787828064]  req@ffff8cd0d48f6d00 x1874671626801792/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828089 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787828064.342668:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828064.342682:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1950
00000100:00080000:1.0:1787828064.342691:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828064.342699:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828064.342735:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828064.342738:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626801792/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828089 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828064.342758:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:1.0:1787828064.342772:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626802176/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828064.342792:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828064.342805:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626801792:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828064.342840:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626801920:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828064.342884:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828064.342912:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828064/real 1787828064]  req@ffff8cd0d48f52c0 x1874671626801920/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828089 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828064.342928:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626801920/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828089 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828064.342939:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828064.342944:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828064.342948:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626801920:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828064.342956:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626802048:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828064.342978:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828064.342998:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828064/real 1787828064]  req@ffff8cd0d48f7c00 x1874671626802048/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828089 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828064.343033:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626802048/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828089 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828064.343065:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828064.343070:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828064.343074:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626802048:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828064.343099:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626802176:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828064.343183:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828064.343199:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828064/real 1787828064]  req@ffff8cd0d48f61c0 x1874671626802176/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828089 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828064.343214:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626802176/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828089 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828064.343225:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828064.343229:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828064.343233:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626802176:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828064.534185:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828064.918276:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828065.558247:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828065.942260:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828066.582203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828066.966371:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828067.606208:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828067.990209:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828068.630185:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828069.014321:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828069.462256:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828069.462268:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828069.462273:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828069.462284:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1955
00000100:00080000:3.0:1787828069.462296:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828069.462326:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828069.462366:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626802304/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828069.462401:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828069.462405:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828069.462407:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828069.462414:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1955
00000100:00080000:3.0:1787828069.462421:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828069.462428:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828069.462458:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828069.462476:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626802432/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828069.462484:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626802304:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787828069.462511:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828069.462515:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828069.462517:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828069.462522:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1955
00000100:00080000:3.0:1787828069.462526:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828069.462530:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828069.462537:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828069.462549:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626802560/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828069.462559:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828069.462562:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000800:00000100:2.0:1787828069.462576:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787828069.462576:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828069.462587:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1955
00000100:00080000:3.0:1787828069.462593:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828069.462598:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828069.462604:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828069.462615:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626802688/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828069.462640:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828069/real 1787828069]  req@ffff8cd0c8eae1c0 x1874671626802304/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828094 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828069.462685:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626802304/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828094 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828069.462696:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828069.462700:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828069.462704:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626802304:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828069.462728:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626802432:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828069.462752:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828069.462764:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828069/real 1787828069]  req@ffff8cd0c8eaf840 x1874671626802432/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828094 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828069.462775:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626802432/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828094 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828069.462782:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828069.462785:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828069.462788:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626802432:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828069.462811:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626802560:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828069.462831:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828069.462842:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828069/real 1787828069]  req@ffff8cd0c8eac780 x1874671626802560/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828094 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828069.462853:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626802560/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828094 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828069.462890:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828069.462893:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828069.462896:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626802560:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828069.462902:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626802688:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828069.462919:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828069.462927:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828069/real 1787828069]  req@ffff8cd0c8ead2c0 x1874671626802688/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828094 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828069.462937:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626802688/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828094 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828069.462944:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828069.462946:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828069.462948:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626802688:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828069.654196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828070.038234:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828070.678146:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828071.062311:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828071.702230:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828072.086149:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828072.726152:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828073.110296:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828073.750230:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828074.134200:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828074.582258:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828074.582307:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828074.582313:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828074.582326:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1960
00000100:00080000:0.0:1787828074.582364:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828074.582374:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828074.582423:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626802816/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828074.582463:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828074.582468:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828074.582471:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828074.582476:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1960
00000100:00080000:0.0:1787828074.582480:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828074.582484:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828074.582497:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828074.582529:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626802944/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828074.582543:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828074.582548:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828074.582552:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828074.582555:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1960
00000100:00080000:0.0:1787828074.582559:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828074.582564:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828074.582570:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828074.582584:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626802816:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787828074.582585:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626803072/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828074.582611:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828074.582623:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828074.582626:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828074.582632:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1960
00000100:00080000:0.0:1787828074.582638:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828074.582643:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828074.582650:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828074.582666:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626803200/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787828074.582711:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828074.582786:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828074/real 1787828074]  req@ffff8cd0c8e9e940 x1874671626802816/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828099 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828074.582809:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626802816/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828099 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828074.582844:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828074.582850:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828074.582855:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626802816:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828074.582865:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626802944:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828074.582898:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828074.582914:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828074/real 1787828074]  req@ffff8cd0c8e9c3c0 x1874671626802944/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828099 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828074.582929:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626802944/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828099 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828074.582943:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828074.582949:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828074.582956:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626802944:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828074.583029:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626803072:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828074.583072:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828074.583125:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828074/real 1787828074]  req@ffff8cd0c8e9ed00 x1874671626803072/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828099 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828074.583142:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626803072/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828099 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828074.583152:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828074.583156:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828074.583160:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626803072:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828074.583179:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626803200:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828074.583207:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828074.583222:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828074/real 1787828074]  req@ffff8cd0c8e9f840 x1874671626803200/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828099 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828074.583235:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626803200/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828099 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828074.583245:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828074.583249:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828074.583252:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626803200:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828074.774171:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828075.158304:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828075.798230:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828076.182190:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828076.822224:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828077.206289:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828077.846206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828078.230265:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828078.870203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828079.254274:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828079.702213:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828079.702225:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828079.702231:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828079.702242:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1965
00000100:00080000:2.0:1787828079.702255:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828079.702264:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828079.702309:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daad00 x1874671626803328/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828079.702344:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828079.702348:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828079.702377:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828079.702382:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1965
00000100:00080000:2.0:1787828079.702388:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828079.702393:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828079.702402:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626803328:192.168.203.145@tcp:250:kworker.0
00000008:00100000:2.0:1787828079.702405:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828079.702430:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626803456/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828079.702445:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828079.702449:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828079.702451:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828079.702456:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1965
00000100:00080000:2.0:1787828079.702464:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828079.702469:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828079.702475:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787828079.702483:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828079.702490:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626803584/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828079.702504:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828079.702508:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828079.702511:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828079.702515:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1965
00000100:00080000:2.0:1787828079.702518:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828079.702523:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828079.702528:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828079.702541:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828079/real 1787828079]  req@ffff8cd0c3daad00 x1874671626803328/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828104 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828079.702553:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626803712/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828079.702593:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daad00 x1874671626803328/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828104 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828079.702614:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828079.702622:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828079.702629:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626803328:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828079.702658:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626803456:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828079.702696:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828079.702719:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828079/real 1787828079]  req@ffff8cd0c3daa1c0 x1874671626803456/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828104 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828079.702753:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626803456/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828104 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828079.702763:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828079.702766:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828079.702770:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626803456:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828079.702777:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626803584:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828079.702804:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828079.702816:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828079/real 1787828079]  req@ffff8cd0c3da92c0 x1874671626803584/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828104 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828079.702829:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626803584/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828104 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828079.702837:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828079.702841:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828079.702844:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626803584:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828079.702850:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626803712:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828079.702865:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828079.702875:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828079/real 1787828079]  req@ffff8cd0c3da9e00 x1874671626803712/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828104 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828079.702886:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626803712/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828104 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828079.702915:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828079.702922:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828079.702927:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626803712:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828079.894201:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828080.278230:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828080.918249:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828081.302189:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828081.942232:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828082.326242:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828082.966211:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828083.350277:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828083.990161:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828084.374191:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828084.822204:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828084.822228:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828084.822234:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828084.822247:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1970
00000100:00080000:2.0:1787828084.822261:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828084.822273:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828084.822322:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626803840/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828084.822365:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828084.822370:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828084.822373:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828084.822378:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1970
00000100:00080000:2.0:1787828084.822383:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828084.822389:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828084.822404:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828084.822419:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626803840:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828084.822433:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626803968/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828084.822461:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828084.822483:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828084.822489:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828084.822499:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1970
00000100:00080000:2.0:1787828084.822509:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1787828084.822510:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787828084.822527:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828084.822539:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828084.822567:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626804096/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828084.822591:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828084/real 1787828084]  req@ffff8cd0c3dabc00 x1874671626803840/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828109 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828084.822592:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828084.822610:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828084.822620:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828084.822634:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1970
00000100:00100000:3.0:1787828084.822639:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626803840/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828109 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828084.822647:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828084.822663:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00080000:3.0:1787828084.822665:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:2.0:1787828084.822673:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00080000:3.0:1787828084.822676:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828084.822688:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626803840:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828084.822705:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626804224/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828084.822727:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626803968:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828084.822766:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828084.822788:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828084/real 1787828084]  req@ffff8cd0c3da8000 x1874671626803968/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828109 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828084.822805:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626803968/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828109 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828084.822815:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828084.822830:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828084.822835:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626803968:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828084.822844:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626804096:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828084.822872:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828084.822886:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828084/real 1787828084]  req@ffff8cd0c3dab840 x1874671626804096/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828109 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828084.822925:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626804096/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828109 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828084.822935:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828084.822939:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828084.822959:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626804096:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828084.822980:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626804224:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828084.823004:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828084.823018:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828084/real 1787828084]  req@ffff8cd0c3daa580 x1874671626804224/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828109 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828084.823033:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626804224/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828109 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828084.823043:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828084.823047:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828084.823051:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626804224:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828085.014216:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828085.398309:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828086.038228:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828086.422219:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828087.062263:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828087.446283:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828088.086191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828088.470237:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828089.110280:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828089.494334:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828089.942270:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828089.942285:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828089.942290:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828089.942303:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1975
00000100:00080000:3.0:1787828089.942317:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828089.942324:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828089.942371:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626804352/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828089.942407:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828089.942411:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828089.942413:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828089.942417:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1975
00000100:00080000:3.0:1787828089.942421:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828089.942427:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828089.942441:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828089.942461:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626804480/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828089.942478:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626804352:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787828089.942480:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828089.942501:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828089.942504:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828089.942509:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1975
00000100:00080000:3.0:1787828089.942512:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828089.942515:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828089.942537:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828089.942552:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626804608/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828089.942560:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828089.942563:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828089.942564:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828089.942567:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1975
00000100:00080000:3.0:1787828089.942572:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828089.942576:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828089.942582:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787828089.942589:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828089.942595:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626804736/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828089.942667:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828089/real 1787828089]  req@ffff8cd0c8eac3c0 x1874671626804352/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828114 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828089.942691:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626804352/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828114 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828089.942711:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828089.942720:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828089.942727:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626804352:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828089.942768:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626804480:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828089.942810:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828089.942829:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828089/real 1787828089]  req@ffff8cd0c8eaf0c0 x1874671626804480/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828114 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828089.942845:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626804480/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828114 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828089.942855:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828089.942859:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828089.942884:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626804480:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828089.942894:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626804608:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828089.942943:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828089.942959:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828089/real 1787828089]  req@ffff8cd0c8eada40 x1874671626804608/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828114 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828089.942974:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626804608/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828114 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828089.943005:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828089.943010:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828089.943014:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626804608:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828089.943023:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626804736:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828089.943051:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828089.943064:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828089/real 1787828089]  req@ffff8cd0c8ead2c0 x1874671626804736/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828114 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828089.943078:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626804736/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828114 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828089.943088:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828089.943091:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828089.943095:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626804736:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828090.134287:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828090.518288:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828091.158214:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828091.542282:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828092.182207:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828092.566234:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828093.206205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828093.590286:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828094.230189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828094.614288:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828095.062219:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828095.062228:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828095.062234:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828095.062248:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1980
00000100:00080000:3.0:1787828095.062263:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828095.062273:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828095.062323:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626804864/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828095.062370:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828095.062375:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828095.062398:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828095.062403:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1980
00000100:00080000:3.0:1787828095.062408:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828095.062412:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828095.062426:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828095.062443:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626804992/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828095.062456:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828095.062461:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828095.062464:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828095.062469:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1980
00000100:00080000:3.0:1787828095.062473:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828095.062476:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828095.062481:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828095.062492:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626805120/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828095.062504:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1787828095.062514:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626804864:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787828095.062514:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828095.062525:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828095.062548:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1980
00000100:00080000:3.0:1787828095.062552:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828095.062556:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828095.062563:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828095.062588:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626805248/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787828095.062639:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828095.062698:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828095/real 1787828095]  req@ffff8cd0c8eac780 x1874671626804864/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828120 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828095.062722:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626804864/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828120 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828095.062757:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828095.062763:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828095.062769:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626804864:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828095.062780:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626804992:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828095.062812:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828095.062860:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828095/real 1787828095]  req@ffff8cd0c8eaf840 x1874671626804992/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828120 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828095.062881:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626804992/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828120 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828095.062892:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828095.062897:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828095.062901:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626804992:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828095.062941:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626805120:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828095.062983:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828095.063007:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828095/real 1787828095]  req@ffff8cd0c8eae1c0 x1874671626805120/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828120 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828095.063060:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626805120/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828120 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828095.063074:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828095.063078:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828095.063150:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626805120:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828095.063176:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626805248:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828095.063220:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828095.063245:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828095/real 1787828095]  req@ffff8cd0c8eae940 x1874671626805248/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828120 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828095.063260:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626805248/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828120 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828095.063271:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828095.063275:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828095.063279:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626805248:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828095.254200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828095.638294:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828096.278146:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828096.662266:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828097.302200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828097.686431:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828098.326218:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828098.710279:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828099.350235:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828099.734261:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828100.182224:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828100.182243:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828100.182251:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828100.182267:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1986
00000100:00080000:2.0:1787828100.182284:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828100.182297:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828100.182365:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626805376/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828100.182406:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828100.182411:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828100.182433:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828100.182440:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1986
00000100:00080000:2.0:1787828100.182447:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828100.182455:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828100.182469:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828100.182493:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626805376:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828100.182494:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab480 x1874671626805504/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828100.182528:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828100.182534:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828100.182536:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828100.182543:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1986
00000100:00080000:2.0:1787828100.182548:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828100.182552:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828100.182560:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828100.182576:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626805632/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828100.182590:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828100.182594:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828100.182597:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828100.182607:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1986
00000800:00000100:3.0:1787828100.182609:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787828100.182614:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828100.182623:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828100.182631:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828100.182660:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626805760/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828100.182677:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828100/real 1787828100]  req@ffff8cd0c3da8780 x1874671626805376/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828125 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828100.182709:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626805376/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828125 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828100.182724:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828100.182730:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828100.182736:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626805376:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828100.182766:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626805504:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828100.182821:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828100.182842:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828100/real 1787828100]  req@ffff8cd0c3dab480 x1874671626805504/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828125 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828100.182859:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab480 x1874671626805504/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828125 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828100.182870:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828100.182875:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828100.182879:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626805504:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828100.182902:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626805632:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828100.182933:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828100.182948:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828100/real 1787828100]  req@ffff8cd0c3daa940 x1874671626805632/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828125 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828100.182963:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626805632/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828125 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828100.182973:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828100.182978:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828100.182982:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626805632:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828100.182989:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626805760:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828100.183008:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828100.183020:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828100/real 1787828100]  req@ffff8cd0c3dab0c0 x1874671626805760/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828125 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828100.183034:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626805760/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828125 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828100.183043:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828100.183047:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828100.183050:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626805760:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828100.374180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828100.758260:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828101.398246:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828101.782323:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828102.422224:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828102.806274:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828103.446229:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828103.830346:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828104.470241:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828104.854255:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828105.302207:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828105.302223:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828105.302230:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828105.302247:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1991
00000100:00080000:2.0:1787828105.302265:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828105.302276:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828105.302341:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626805888/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828105.302385:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828105.302390:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828105.302392:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828105.302397:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1991
00000100:00080000:2.0:1787828105.302402:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828105.302407:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828105.302422:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828105.302439:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626806016/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828105.302454:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828105.302488:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828105.302492:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828105.302504:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1991
00000100:00100000:3.0:1787828105.302506:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626805888:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828105.302512:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828105.302527:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828105.302536:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828105.302555:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626806144/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828105.302572:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828105.302577:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828105.302579:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828105.302584:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1991
00000100:00080000:2.0:1787828105.302589:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828105.302595:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828105.302601:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828105.302621:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626806272/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828105.302660:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828105.302737:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828105/real 1787828105]  req@ffff8cd0c3da9a40 x1874671626805888/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828130 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828105.302792:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626805888/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828130 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828105.302809:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828105.302814:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828105.302820:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626805888:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828105.302838:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626806016:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828105.302883:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828105.302900:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828105/real 1787828105]  req@ffff8cd0c3da8f00 x1874671626806016/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828130 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828105.302917:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626806016/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828130 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828105.302928:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828105.302932:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828105.302936:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626806016:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828105.302968:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626806144:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828105.303009:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828105.303023:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828105/real 1787828105]  req@ffff8cd0c3da9680 x1874671626806144/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828130 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828105.303059:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626806144/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828130 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828105.303075:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828105.303082:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828105.303155:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626806144:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828105.303170:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626806272:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828105.303208:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828105.303241:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828105/real 1787828105]  req@ffff8cd0c3da83c0 x1874671626806272/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828130 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828105.303258:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626806272/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828130 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828105.303268:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828105.303273:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828105.303277:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626806272:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828105.494224:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828105.878372:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828106.518297:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828106.902257:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828107.542198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828107.926364:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828108.566176:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828108.950254:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828109.590177:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828109.974332:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828110.422211:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828110.422224:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828110.422230:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828110.422243:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 1996
00000100:00080000:3.0:1787828110.422259:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828110.422296:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828110.422346:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626806400/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828110.422390:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828110.422396:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828110.422399:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828110.422407:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1996
00000100:00080000:3.0:1787828110.422413:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828110.422420:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828110.422433:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828110.422435:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626806400:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828110.422451:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626806528/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828110.422464:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828110.422468:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828110.422470:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828110.422473:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 1996
00000100:00080000:3.0:1787828110.422477:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828110.422481:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828110.422486:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828110.422498:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626806656/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828110.422507:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828110.422510:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828110.422521:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828110.422526:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 1996
00000100:00080000:3.0:1787828110.422531:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828110.422535:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828110.422540:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787828110.422554:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828110.422555:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626806784/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828110.422661:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828110/real 1787828110]  req@ffff8cd0c8eacf00 x1874671626806400/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828135 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828110.422690:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626806400/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828135 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828110.422706:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828110.422712:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828110.422718:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626806400:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828110.422809:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626806528:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828110.422836:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828110.422858:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828110/real 1787828110]  req@ffff8cd0c8eac000 x1874671626806528/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828135 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828110.422870:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626806528/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828135 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828110.422879:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828110.422882:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828110.422885:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626806528:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828110.422890:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626806656:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828110.422909:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828110.422918:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828110/real 1787828110]  req@ffff8cd0c8eae940 x1874671626806656/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828135 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828110.422928:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626806656/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828135 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828110.422971:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828110.422974:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828110.422977:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626806656:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828110.422983:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626806784:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828110.423004:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828110.423014:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828110/real 1787828110]  req@ffff8cd0c8eae1c0 x1874671626806784/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828135 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828110.423024:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626806784/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828135 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828110.423031:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828110.423033:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828110.423036:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626806784:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828110.614221:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828110.998235:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828111.638232:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828112.022362:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828112.662209:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828113.046280:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828113.686327:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828114.070266:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828114.710187:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828115.094189:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828115.542229:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828115.542251:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828115.542258:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828115.542284:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2001
00000100:00080000:3.0:1787828115.542304:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828115.542315:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828115.542380:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626806912/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828115.542434:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828115.542439:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828115.542441:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828115.542447:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2001
00000100:00080000:3.0:1787828115.542451:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828115.542456:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828115.542471:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828115.542518:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626807040/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828115.542536:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828115.542541:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828115.542546:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1787828115.542550:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626806912:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787828115.542557:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2001
00000100:00080000:3.0:1787828115.542569:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828115.542574:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828115.542582:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828115.542603:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626807168/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828115.542616:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828115.542621:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828115.542623:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828115.542629:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2001
00000100:00080000:3.0:1787828115.542649:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828115.542655:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828115.542662:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828115.542679:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626807296/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828115.542789:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626807040:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828115.542855:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626807168:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828115.542907:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626807296:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828115.734173:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828116.118323:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828116.758268:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828117.142258:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828117.782214:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828118.166378:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:1.0:1787828118.614570:0:2343:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:1.0:1787828118.614640:0:2343:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1787828118.614698:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787828118.614757:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787828118.614792:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787828118.614814:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:2.0:1787828118.614864:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828115/real 1787828118]  req@ffff8cd0c8eac780 x1874671626807040/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828140 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828118.614916:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626807040/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828140 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828118.614942:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828118.614952:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828118.614960:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626807040:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828118.614974:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828115/real 1787828118]  req@ffff8cd0c8ead2c0 x1874671626807168/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828140 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828118.614998:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626807168/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828140 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828118.615014:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828118.615021:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828118.615028:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626807168:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828118.615042:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828115/real 1787828118]  req@ffff8cd0c8eada40 x1874671626807296/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828140 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828118.615069:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626807296/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828140 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828118.615132:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828118.615173:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828118.615181:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626807296:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828118.615195:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828115/real 1787828118]  req@ffff8cd0c8eaf840 x1874671626806912/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828140 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828118.615224:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626806912/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828140 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828118.615241:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828118.615249:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828118.615257:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626806912:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828118.806217:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828119.190228:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828119.830200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828120.214235:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828120.662216:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828120.662229:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828120.662236:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828120.662251:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2006
00000100:00080000:2.0:1787828120.662282:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828120.662296:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828120.662372:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8b40 x1874671626807424/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828120.662435:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828120.662443:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828120.662447:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828120.662466:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2006
00000100:00080000:2.0:1787828120.662474:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828120.662519:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828120.662537:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626807424:192.168.203.145@tcp:250:kworker.0
00000008:00100000:2.0:1787828120.662539:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828120.662576:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626807552/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828120.662605:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828120.662613:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828120.662618:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828120.662626:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2006
00000100:00080000:2.0:1787828120.662636:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828120.662647:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:3.0:1787828120.662671:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:2.0:1787828120.662691:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828120.662722:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626807680/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828120.662735:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828120/real 1787828120]  req@ffff8cd0c3da8b40 x1874671626807424/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828145 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828120.662756:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828120.662769:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828120.662774:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1787828120.662786:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8b40 x1874671626807424/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828145 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828120.662792:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2006
00000100:00080000:2.0:1787828120.662806:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828120.662816:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828120.662820:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00080000:3.0:1787828120.662830:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000008:00100000:2.0:1787828120.662839:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828120.662841:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626807424:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828120.662879:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626807808/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828120.662896:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626807552:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828120.663073:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828120.663125:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828120/real 1787828120]  req@ffff8cd0c3da83c0 x1874671626807552/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828145 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828120.663144:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626807552/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828145 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828120.663154:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828120.663158:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828120.663162:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626807552:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828120.663171:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626807680:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828120.663202:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828120.663217:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828120/real 1787828120]  req@ffff8cd0c3da9680 x1874671626807680/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828145 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828120.663252:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626807680/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828145 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828120.663262:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828120.663267:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828120.663271:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626807680:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828120.663292:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626807808:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828120.663333:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828120.663349:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828120/real 1787828120]  req@ffff8cd0c3da8f00 x1874671626807808/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828145 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828120.663363:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626807808/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828145 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828120.663372:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828120.663376:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828120.663380:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626807808:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828120.854237:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828121.238251:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828121.878190:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828122.262352:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828122.902192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828123.286223:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828123.926228:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828124.310333:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828124.950197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828125.334202:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828125.782233:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828125.782248:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828125.782254:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828125.782269:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2011
00000100:00080000:3.0:1787828125.782285:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828125.782294:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828125.782378:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626807936/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828125.782435:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828125.782443:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828125.782446:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828125.782452:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2011
00000100:00080000:3.0:1787828125.782459:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828125.782467:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828125.782482:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828125.782496:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626807936:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828125.782508:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626808064/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828125.782527:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828125.782532:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828125.782535:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828125.782540:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2011
00000100:00080000:3.0:1787828125.782545:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828125.782549:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828125.782555:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828125.782573:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626808192/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828125.782587:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828125.782592:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828125.782594:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828125.782638:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2011
00000800:00000100:2.0:1787828125.782648:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787828125.782652:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828125.782662:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828125.782672:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828125.782692:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626808320/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828125.782746:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828125/real 1787828125]  req@ffff8cd0c8eaf0c0 x1874671626807936/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828150 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828125.782775:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626807936/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828150 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828125.782791:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828125.782873:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828125.782879:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626807936:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828125.782921:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626808064:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828125.782958:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828125.782979:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828125/real 1787828125]  req@ffff8cd0c8eac3c0 x1874671626808064/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828150 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828125.782996:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626808064/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828150 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828125.783007:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828125.783011:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828125.783015:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626808064:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828125.783023:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626808192:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828125.783073:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828125.783118:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828125/real 1787828125]  req@ffff8cd0c8eaf840 x1874671626808192/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828150 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828125.783134:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626808192/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828150 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828125.783143:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828125.783147:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828125.783172:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626808192:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828125.783181:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626808320:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828125.783212:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828125.783225:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828125/real 1787828125]  req@ffff8cd0c8eada40 x1874671626808320/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828150 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828125.783239:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626808320/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828150 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828125.783249:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828125.783252:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828125.783256:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626808320:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828125.974243:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828126.358212:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828126.998143:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828127.382230:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828128.022135:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828128.406270:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828129.046196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828129.430293:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828130.070203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828130.454267:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828130.902156:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828130.902166:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828130.902171:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828130.902182:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2016
00000100:00080000:3.0:1787828130.902193:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828130.902200:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828130.902243:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626808448/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828130.902268:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828130.902272:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828130.902273:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828130.902277:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2016
00000100:00080000:3.0:1787828130.902279:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828130.902283:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828130.902292:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828130.902318:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626808448:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828130.902321:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626808576/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828130.902337:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828130.902342:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828130.902344:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828130.902348:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2016
00000100:00080000:3.0:1787828130.902353:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828130.902358:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828130.902365:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828130.902388:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626808704/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828130.902396:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828130.902399:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828130.902401:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828130.902404:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2016
00000100:00080000:3.0:1787828130.902409:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1787828130.902414:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787828130.902417:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828130.902422:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828130.902432:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626808832/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828130.902474:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828130/real 1787828130]  req@ffff8cd0c8ead2c0 x1874671626808448/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828155 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828130.902497:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626808448/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828155 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828130.902510:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828130.902514:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828130.902519:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626808448:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828130.902546:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626808576:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828130.902567:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828130.902591:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828130/real 1787828130]  req@ffff8cd0c8eac780 x1874671626808576/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828155 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828130.902602:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626808576/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828155 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828130.902609:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828130.902611:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828130.902614:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626808576:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828130.902618:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626808704:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828130.902635:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828130.902643:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828130/real 1787828130]  req@ffff8cd0c8eae1c0 x1874671626808704/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828155 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828130.902663:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626808704/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828155 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828130.902669:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828130.902672:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828130.902674:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626808704:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828130.902678:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626808832:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828130.902692:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828130.902699:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828130/real 1787828130]  req@ffff8cd0c8eae940 x1874671626808832/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828155 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828130.902706:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626808832/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828155 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828130.902720:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828130.902723:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828130.902725:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626808832:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828131.094163:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828131.478269:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828132.118230:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828132.502291:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828133.142200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828133.526331:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828134.166222:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828134.550271:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828135.190231:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828135.574163:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828136.022239:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828136.022253:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828136.022291:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828136.022308:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2021
00000100:00080000:3.0:1787828136.022325:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828136.022336:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828136.022395:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626808960/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828136.022453:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828136.022458:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828136.022461:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828136.022466:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2021
00000100:00080000:3.0:1787828136.022471:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828136.022476:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828136.022490:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828136.022505:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626809088/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828136.022519:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828136.022524:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828136.022526:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828136.022530:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2021
00000100:00080000:3.0:1787828136.022533:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828136.022538:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828136.022567:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828136.022588:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626809216/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828136.022622:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626808960:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787828136.022627:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828136.022638:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828136.022640:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828136.022648:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2021
00000100:00080000:3.0:1787828136.022654:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828136.022660:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828136.022668:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828136.022686:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626809344/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787828136.022747:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828136.022816:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828136/real 1787828136]  req@ffff8cd0c8eac000 x1874671626808960/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828161 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828136.022865:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626808960/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828161 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828136.022881:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828136.022887:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828136.022893:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626808960:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828136.022903:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626809088:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828136.022935:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828136.022951:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828136/real 1787828136]  req@ffff8cd0c8eacf00 x1874671626809088/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828161 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828136.022966:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626809088/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828161 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828136.022977:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828136.022981:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828136.022985:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626809088:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828136.023020:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626809216:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828136.023042:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828136.023054:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828136/real 1787828136]  req@ffff8cd0c8eae940 x1874671626809216/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828161 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828136.023068:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626809216/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828161 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828136.023162:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828136.023167:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828136.023172:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626809216:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828136.023180:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626809344:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828136.023212:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828136.023228:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828136/real 1787828136]  req@ffff8cd0c8eae1c0 x1874671626809344/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828161 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828136.023242:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626809344/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828161 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828136.023252:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828136.023256:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828136.023260:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626809344:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828136.214223:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828136.598315:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828137.238308:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828137.622262:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828138.262257:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828138.646333:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828139.286233:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828139.670297:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828140.310238:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828140.694300:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828141.142291:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828141.142304:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828141.142310:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828141.142323:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2026
00000100:00080000:1.0:1787828141.142337:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828141.142345:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828141.142397:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626809472/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828141.142430:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828141.142433:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828141.142435:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828141.142438:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2026
00000100:00080000:1.0:1787828141.142441:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828141.142445:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828141.142456:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828141.142468:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626809600/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828141.142484:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626809472:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787828141.142503:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828141.142524:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787828141.142527:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828141.142532:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2026
00000100:00080000:1.0:1787828141.142536:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828141.142541:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828141.142548:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828141.142575:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626809728/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828141.142583:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828141.142586:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828141.142588:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828141.142591:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2026
00000100:00080000:1.0:1787828141.142594:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828141.142597:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828141.142601:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787828141.142603:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787828141.142612:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626809856/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828141.142655:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828141/real 1787828141]  req@ffff8cd0d48f70c0 x1874671626809472/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828166 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828141.142673:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626809472/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828166 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828141.142690:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828141.142694:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828141.142697:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626809472:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828141.142723:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626809600:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828141.142752:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828141.142779:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828141/real 1787828141]  req@ffff8cd0d48f43c0 x1874671626809600/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828166 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828141.142787:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626809600/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828166 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828141.142793:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828141.142795:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828141.142797:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626809600:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828141.142802:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626809728:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828141.142815:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828141.142821:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828141/real 1787828141]  req@ffff8cd0d48f5a40 x1874671626809728/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828166 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828141.142845:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626809728/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828166 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828141.142851:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828141.142853:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828141.142855:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626809728:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828141.142865:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626809856:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828141.142880:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828141.142888:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828141/real 1787828141]  req@ffff8cd0d48f7840 x1874671626809856/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828166 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828141.142896:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626809856/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828166 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828141.142902:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828141.142912:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828141.142916:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626809856:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828141.334166:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828141.718195:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828142.358158:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828142.742361:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828143.382267:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828143.766218:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828144.406143:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828144.790244:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828145.430200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828145.814373:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828146.262271:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828146.262283:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828146.262289:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828146.262302:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2032
00000100:00080000:1.0:1787828146.262344:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828146.262354:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828146.262403:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626809984/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828146.262445:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828146.262450:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828146.262453:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828146.262457:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2032
00000100:00080000:1.0:1787828146.262462:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828146.262466:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828146.262500:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828146.262535:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626810112/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828146.262550:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828146.262554:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787828146.262557:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828146.262565:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2032
00000100:00080000:1.0:1787828146.262571:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1787828146.262571:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626809984:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787828146.262604:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828146.262612:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828146.262636:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626810240/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828146.262656:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828146.262674:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828146.262680:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828146.262688:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2032
00000100:00080000:1.0:1787828146.262712:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1787828146.262727:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787828146.262728:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828146.262747:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828146.262796:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626810368/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828146.262814:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828146/real 1787828146]  req@ffff8cd0d48f4b40 x1874671626809984/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828171 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828146.262871:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626809984/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828171 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828146.262885:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828146.262893:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828146.262905:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626809984:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828146.262972:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626810112:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828146.263027:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828146.263044:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828146/real 1787828146]  req@ffff8cd0d48f6580 x1874671626810112/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828171 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828146.263057:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626810112/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828171 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828146.263067:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828146.263070:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828146.263074:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626810112:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828146.263081:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626810240:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828146.263171:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828146.263190:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828146/real 1787828146]  req@ffff8cd0d48f7480 x1874671626810240/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828171 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828146.263214:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626810240/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828171 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828146.263233:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828146.263240:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828146.263247:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626810240:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828146.263260:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626810368:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828146.263306:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828146.263330:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828146/real 1787828146]  req@ffff8cd0d48f4000 x1874671626810368/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828171 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828146.263373:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626810368/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828171 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828146.263391:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828146.263398:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828146.263405:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626810368:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828146.454241:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828146.838392:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828147.478273:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828147.862239:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828148.502238:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828148.886313:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828149.526210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828149.910242:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828150.550258:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828150.934334:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828151.382248:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828151.382260:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828151.382277:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828151.382290:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2037
00000100:00080000:0.0:1787828151.382304:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828151.382314:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828151.382371:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626810496/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828151.382410:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828151.382414:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828151.382417:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828151.382421:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2037
00000100:00080000:0.0:1787828151.382425:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828151.382471:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828151.382495:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828151.382517:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626810624/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828151.382540:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828151.382547:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828151.382551:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828151.382562:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2037
00000100:00100000:2.0:1787828151.382574:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626810496:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828151.382579:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828151.382598:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828151.382613:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828151.382646:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626810752/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828151.382663:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828151.382669:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828151.382672:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828151.382677:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2037
00000100:00080000:0.0:1787828151.382683:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828151.382689:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828151.382698:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787828151.382719:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787828151.382720:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626810880/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828151.382801:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828151/real 1787828151]  req@ffff8cd0c8e9d680 x1874671626810496/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828176 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828151.382825:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626810496/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828176 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828151.382840:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828151.382845:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828151.382850:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626810496:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828151.382861:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626810624:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828151.382887:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828151.382901:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828151/real 1787828151]  req@ffff8cd0c8e9f480 x1874671626810624/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828176 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828151.382936:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626810624/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828176 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828151.382945:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828151.382949:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828151.382953:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626810624:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828151.382990:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626810752:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828151.383015:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828151.383026:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828151/real 1787828151]  req@ffff8cd0c8e9cb40 x1874671626810752/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828176 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828151.383039:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626810752/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828176 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828151.383047:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828151.383050:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828151.383053:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626810752:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828151.383060:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626810880:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828151.383077:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828151.383151:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828151/real 1787828151]  req@ffff8cd0c8e9c000 x1874671626810880/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828176 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828151.383165:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626810880/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828176 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828151.383189:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828151.383194:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828151.383197:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626810880:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828151.574235:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828151.958265:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828152.598259:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828152.982238:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828153.622173:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828154.006256:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828154.646217:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828155.030337:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828155.670263:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828156.054194:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828156.502187:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828156.502206:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828156.502213:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828156.502245:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2042
00000100:00080000:0.0:1787828156.502261:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828156.502271:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828156.502344:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d2c0 x1874671626811008/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828156.502382:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828156.502386:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828156.502388:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828156.502392:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2042
00000100:00080000:0.0:1787828156.502396:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828156.502420:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828156.502436:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828156.502467:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626811008:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787828156.502474:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626811136/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828156.502487:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828156.502491:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828156.502493:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828156.502497:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2042
00000100:00080000:0.0:1787828156.502501:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828156.502505:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828156.502526:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828156.502540:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626811264/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828156.502550:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828156.502570:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828156.502571:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828156.502574:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2042
00000100:00080000:0.0:1787828156.502578:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1787828156.502583:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787828156.502584:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828156.502590:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828156.502637:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626811392/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828156.502657:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828156/real 1787828156]  req@ffff8cd0c8e9d2c0 x1874671626811008/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828181 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828156.502681:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d2c0 x1874671626811008/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828181 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828156.502693:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828156.502697:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828156.502701:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626811008:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828156.502747:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626811136:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828156.502766:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828156.502776:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828156/real 1787828156]  req@ffff8cd0c8e9c000 x1874671626811136/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828181 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828156.502787:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626811136/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828181 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828156.502800:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828156.502803:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828156.502805:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626811136:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828156.502810:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626811264:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828156.502826:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828156.502833:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828156/real 1787828156]  req@ffff8cd0c8e9cb40 x1874671626811264/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828181 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828156.502841:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626811264/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828181 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828156.502846:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828156.502848:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828156.502850:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626811264:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828156.502854:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626811392:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828156.502866:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828156.502872:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828156/real 1787828156]  req@ffff8cd0c8e9f480 x1874671626811392/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828181 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828156.502880:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626811392/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828181 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828156.502884:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828156.502886:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828156.502893:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626811392:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828156.694328:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828157.078297:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828157.718221:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828158.102305:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828158.742214:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828159.126309:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828159.766201:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828160.150194:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828160.790197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828161.174262:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828161.622269:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828161.622281:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828161.622296:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828161.622308:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2047
00000100:00080000:3.0:1787828161.622345:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828161.622351:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828161.622420:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626811520/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828161.622454:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828161.622458:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828161.622460:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828161.622465:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2047
00000100:00080000:3.0:1787828161.622468:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828161.622471:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828161.622482:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828161.622497:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626811648/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828161.622502:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626811520:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787828161.622511:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828161.622515:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828161.622516:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828161.622520:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2047
00000100:00080000:3.0:1787828161.622522:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828161.622524:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828161.622528:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828161.622536:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626811776/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828161.622543:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828161.622551:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828161.622553:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828161.622556:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2047
00000100:00080000:3.0:1787828161.622568:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828161.622571:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828161.622575:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828161.622584:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626811904/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787828161.622591:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828161.622639:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828161/real 1787828161]  req@ffff8cd0c8eac780 x1874671626811520/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828186 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828161.622655:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626811520/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828186 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828161.622664:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828161.622667:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828161.622670:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626811520:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828161.622694:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626811648:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828161.622709:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828161.622718:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828161/real 1787828161]  req@ffff8cd0c8ead2c0 x1874671626811648/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828186 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828161.622725:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626811648/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828186 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828161.622731:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828161.622733:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828161.622742:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626811648:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828161.622747:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626811776:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828161.622762:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828161.622769:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828161/real 1787828161]  req@ffff8cd0c8eada40 x1874671626811776/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828186 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828161.622776:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626811776/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828186 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828161.622781:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828161.622783:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828161.622785:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626811776:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828161.622788:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626811904:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828161.622799:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828161.622805:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828161/real 1787828161]  req@ffff8cd0c8eaf840 x1874671626811904/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828186 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828161.622812:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626811904/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828186 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828161.622817:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828161.622818:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828161.622820:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626811904:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828161.814159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828162.198303:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828162.838224:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828163.222422:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828163.862254:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828164.246300:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828164.886220:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828165.270249:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828165.910212:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828166.294321:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828166.742221:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828166.742230:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828166.742235:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828166.742243:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2052
00000100:00080000:2.0:1787828166.742254:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828166.742261:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828166.742296:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626812032/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828166.742339:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828166.742343:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828166.742344:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828166.742347:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2052
00000100:00080000:2.0:1787828166.742350:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828166.742354:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828166.742380:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828166.742392:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626812160/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828166.742402:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828166.742405:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828166.742407:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828166.742419:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2052
00000100:00100000:3.0:1787828166.742420:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626812032:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828166.742423:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828166.742444:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828166.742449:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828166.742463:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626812288/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828166.742472:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828166.742476:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828166.742478:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828166.742481:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2052
00000100:00080000:2.0:1787828166.742504:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1787828166.742510:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787828166.742514:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828166.742522:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828166.742535:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab480 x1874671626812416/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828166.742568:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828166/real 1787828166]  req@ffff8cd0c3da9a40 x1874671626812032/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828191 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828166.742614:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626812032/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828191 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828166.742629:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828166.742635:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828166.742641:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626812032:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828166.742651:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626812160:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828166.742681:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828166.742696:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828166/real 1787828166]  req@ffff8cd0c3dab0c0 x1874671626812160/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828191 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828166.742770:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626812160/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828191 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828166.742788:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828166.742794:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828166.742802:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626812160:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828166.742843:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626812288:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828166.742896:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828166.742906:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828166/real 1787828166]  req@ffff8cd0c3daa940 x1874671626812288/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828191 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828166.742915:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626812288/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828191 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828166.742921:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828166.742924:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828166.742926:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626812288:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828166.742930:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626812416:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828166.742952:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828166.742959:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828166/real 1787828166]  req@ffff8cd0c3dab480 x1874671626812416/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828191 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828166.742967:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab480 x1874671626812416/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828191 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828166.742983:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828166.742985:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828166.742988:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626812416:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828166.934226:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828167.318255:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828167.958173:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828168.342219:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828168.982360:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828169.366291:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828170.006231:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828170.390243:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828171.030150:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828171.414385:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828171.862262:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828171.862273:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828171.862278:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828171.862288:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2057
00000100:00080000:3.0:1787828171.862323:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828171.862333:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828171.862403:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626812544/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828171.862433:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828171.862437:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828171.862439:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828171.862443:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2057
00000100:00080000:3.0:1787828171.862447:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828171.862451:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828171.862462:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828171.862475:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626812672/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828171.862510:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626812544:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787828171.862532:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828171.862536:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828171.862539:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828171.862544:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2057
00000100:00080000:3.0:1787828171.862549:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828171.862562:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828171.862569:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828171.862584:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626812800/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828171.862611:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828171.862617:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000800:00000100:2.0:1787828171.862618:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787828171.862620:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828171.862627:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2057
00000100:00080000:3.0:1787828171.862632:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828171.862636:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828171.862642:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828171.862653:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626812928/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828171.862667:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828171/real 1787828171]  req@ffff8cd0c8eac3c0 x1874671626812544/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828196 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828171.862698:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626812544/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828196 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828171.862723:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828171.862728:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828171.862732:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626812544:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828171.862767:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626812672:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828171.862791:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828171.862802:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828171/real 1787828171]  req@ffff8cd0c8eaf0c0 x1874671626812672/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828196 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828171.862813:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626812672/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828196 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828171.862820:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828171.862823:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828171.862825:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626812672:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828171.862831:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626812800:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828171.862846:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828171.862853:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828171/real 1787828171]  req@ffff8cd0c8eaf840 x1874671626812800/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828196 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828171.862863:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626812800/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828196 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828171.862870:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828171.862872:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828171.862875:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626812800:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828171.862886:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626812928:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828171.862904:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828171.862913:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828171/real 1787828171]  req@ffff8cd0c8eada40 x1874671626812928/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828196 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828171.862923:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626812928/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828196 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828171.862930:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828171.862932:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828171.862934:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626812928:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828172.054146:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828172.438220:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828173.078149:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828173.462222:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828174.102192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828174.486291:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828175.126210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828175.510412:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828176.150203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828176.534254:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828176.982370:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828176.982382:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828176.982388:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828176.982414:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2062
00000100:00080000:1.0:1787828176.982435:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828176.982445:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828176.982509:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626813056/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828176.982547:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828176.982552:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828176.982555:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828176.982559:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2062
00000100:00080000:1.0:1787828176.982563:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828176.982568:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828176.982581:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828176.982596:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626813184/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828176.982633:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1787828176.982639:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626813056:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787828176.982640:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787828176.982647:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828176.982661:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2062
00000100:00080000:1.0:1787828176.982666:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828176.982671:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828176.982679:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828176.982697:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626813312/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828176.982710:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828176.982715:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828176.982718:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828176.982722:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2062
00000100:00080000:1.0:1787828176.982727:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828176.982731:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828176.982737:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787828176.982752:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787828176.982759:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626813440/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828176.982829:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828176/real 1787828176]  req@ffff8cd0d48f4000 x1874671626813056/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828201 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828176.982855:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626813056/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828201 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828176.982869:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828176.982874:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828176.982879:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626813056:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828176.982923:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626813184:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828176.982956:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828176.982972:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828176/real 1787828176]  req@ffff8cd0d48f7480 x1874671626813184/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828201 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828176.982988:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626813184/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828201 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828176.982999:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828176.983003:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828176.983007:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626813184:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828176.983014:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626813312:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828176.983034:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828176.983045:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828176/real 1787828176]  req@ffff8cd0d48f6580 x1874671626813312/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828201 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828176.983059:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626813312/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828201 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828176.983069:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828176.983073:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828176.983076:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626813312:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828176.983124:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626813440:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828176.983160:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828176.983174:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828176/real 1787828176]  req@ffff8cd0d48f4b40 x1874671626813440/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828201 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828176.983201:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626813440/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828201 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828176.983212:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828176.983216:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828176.983220:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626813440:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828177.174192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828177.558329:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828178.198247:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828178.582244:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828179.222270:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828179.606387:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828180.246188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828180.630328:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828181.270156:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828181.654348:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828182.102273:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828182.102313:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828182.102319:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828182.102331:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2067
00000100:00080000:3.0:1787828182.102346:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828182.102356:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828182.102406:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626813568/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828182.102449:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828182.102456:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828182.102461:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828182.102479:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2067
00000100:00080000:3.0:1787828182.102502:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828182.102506:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828182.102517:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828182.102532:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626813696/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828182.102545:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1787828182.102567:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626813568:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787828182.102569:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828182.102579:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828182.102592:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2067
00000100:00080000:3.0:1787828182.102601:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828182.102609:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828182.102624:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828182.102684:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626813824/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828182.102696:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828182.102699:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828182.102702:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828182.102707:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2067
00000100:00080000:3.0:1787828182.102713:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1787828182.102718:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626813696:192.168.203.145@tcp:38:kworker.0
00000100:00080000:3.0:1787828182.102719:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828182.102754:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828182.102783:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626813952/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828182.102803:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626813824:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828182.102847:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626813952:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828182.294210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828182.678208:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828183.318234:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828183.702335:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828184.342200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828184.726194:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:1.0:1787828185.174348:0:2345:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:1.0:1787828185.174383:0:2345:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1787828185.174403:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787828185.174434:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787828185.174461:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787828185.174470:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:2.0:1787828185.174475:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828182/real 1787828185]  req@ffff8cd0c8eac780 x1874671626813696/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828207 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828185.174501:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626813696/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828207 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828185.174511:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828185.174520:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828185.174523:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626813696:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828185.174529:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828182/real 1787828185]  req@ffff8cd0c8eae1c0 x1874671626813824/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828207 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828185.174544:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626813824/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828207 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828185.174554:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828185.174558:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828185.174561:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626813824:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828185.174569:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828182/real 1787828185]  req@ffff8cd0c8ead2c0 x1874671626813568/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828207 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828185.174582:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626813568/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828207 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828185.174591:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828185.174595:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828185.174598:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626813568:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828185.174623:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828182/real 1787828185]  req@ffff8cd0c8eae940 x1874671626813952/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828207 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828185.174635:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626813952/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828207 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828185.174643:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828185.174654:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828185.174658:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626813952:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828185.366191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828185.750272:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828186.390190:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828186.774204:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828187.222223:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828187.222232:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828187.222235:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828187.222244:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2073
00000100:00080000:3.0:1787828187.222253:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828187.222260:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828187.222290:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626814080/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828187.222314:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828187.222316:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828187.222318:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828187.222334:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2073
00000100:00080000:3.0:1787828187.222339:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828187.222343:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828187.222351:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828187.222361:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626814080:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828187.222365:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626814208/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828187.222375:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828187.222377:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828187.222379:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828187.222382:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2073
00000100:00080000:3.0:1787828187.222384:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828187.222386:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828187.222390:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828187.222398:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626814336/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828187.222405:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828187.222408:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828187.222409:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828187.222412:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2073
00000100:00080000:3.0:1787828187.222414:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828187.222417:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:2.0:1787828187.222422:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:3.0:1787828187.222422:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828187.222437:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626814464/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828187.222458:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828187/real 1787828187]  req@ffff8cd0c8eacf00 x1874671626814080/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828212 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828187.222472:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626814080/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828212 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828187.222482:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828187.222485:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828187.222488:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626814080:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828187.222514:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626814208:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828187.222530:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828187.222540:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828187/real 1787828187]  req@ffff8cd0c8eac000 x1874671626814208/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828212 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828187.222553:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626814208/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828212 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828187.222563:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828187.222567:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828187.222571:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626814208:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828187.222584:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626814336:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828187.222608:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828187.222620:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828187/real 1787828187]  req@ffff8cd0c8eae940 x1874671626814336/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828212 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828187.222633:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626814336/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828212 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828187.222642:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828187.222645:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828187.222649:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626814336:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828187.222655:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626814464:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828187.222683:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828187.222696:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828187/real 1787828187]  req@ffff8cd0c8ead2c0 x1874671626814464/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828212 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828187.222708:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626814464/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828212 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828187.222716:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828187.222719:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828187.222723:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626814464:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828187.414181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828187.798220:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828188.438196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828188.822307:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828189.462271:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828189.846305:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828190.486182:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828190.870221:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828191.510186:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828191.894409:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828192.342208:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828192.342218:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828192.342222:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828192.342233:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2078
00000100:00080000:3.0:1787828192.342246:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828192.342252:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828192.342296:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626814592/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828192.342324:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828192.342327:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828192.342328:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828192.342331:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2078
00000100:00080000:3.0:1787828192.342333:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828192.342336:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828192.342345:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828192.342353:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626814720/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828192.342360:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828192.342363:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828192.342364:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828192.342386:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2078
00000100:00080000:3.0:1787828192.342389:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828192.342393:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828192.342400:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828192.342409:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626814592:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828192.342411:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626814848/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828192.342433:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828192.342436:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828192.342438:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828192.342445:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2078
00000100:00080000:3.0:1787828192.342448:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828192.342451:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828192.342454:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828192.342461:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626814976/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787828192.342493:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828192.342533:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828192/real 1787828192]  req@ffff8cd0c8eae1c0 x1874671626814592/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828217 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828192.342547:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626814592/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828217 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828192.342557:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828192.342560:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828192.342563:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626814592:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828192.342569:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626814720:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828192.342584:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828192.342592:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828192/real 1787828192]  req@ffff8cd0c8eac780 x1874671626814720/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828217 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828192.342600:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626814720/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828217 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828192.342605:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828192.342607:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828192.342609:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626814720:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828192.342630:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626814848:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828192.342642:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828192.342649:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828192/real 1787828192]  req@ffff8cd0c8eada40 x1874671626814848/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828217 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828192.342671:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626814848/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828217 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828192.342676:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828192.342678:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828192.342681:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626814848:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828192.342685:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626814976:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828192.342698:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828192.342704:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828192/real 1787828192]  req@ffff8cd0c8eaf840 x1874671626814976/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828217 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828192.342711:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626814976/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828217 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828192.342716:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828192.342718:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828192.342720:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626814976:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828192.534178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828192.918230:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828193.558181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828193.942242:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828194.582155:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828194.966177:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828195.606188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828195.990231:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828196.630151:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828197.014177:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828197.462200:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828197.462208:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828197.462212:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828197.462234:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2083
00000100:00080000:2.0:1787828197.462243:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828197.462250:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828197.462280:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626815104/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828197.462304:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828197.462307:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828197.462308:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828197.462311:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2083
00000100:00080000:2.0:1787828197.462313:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828197.462316:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828197.462324:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828197.462333:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626815232/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828197.462342:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787828197.462345:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626815104:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828197.462346:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828197.462348:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828197.462355:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2083
00000100:00080000:2.0:1787828197.462358:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828197.462361:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828197.462364:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828197.462372:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626815360/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828197.462389:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828197.462391:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828197.462393:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828197.462396:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2083
00000100:00080000:2.0:1787828197.462400:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828197.462403:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828197.462406:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787828197.462413:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828197.462416:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626815488/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828197.462457:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828197/real 1787828197]  req@ffff8cd0c3da8780 x1874671626815104/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828222 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828197.462476:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626815104/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828222 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828197.462501:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828197.462505:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828197.462509:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626815104:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828197.462531:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626815232:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828197.462553:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828197.462565:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828197/real 1787828197]  req@ffff8cd0c3daa580 x1874671626815232/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828222 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828197.462578:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626815232/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828222 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828197.462587:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828197.462590:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828197.462594:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626815232:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828197.462600:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626815360:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828197.462617:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828197.462627:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828197/real 1787828197]  req@ffff8cd0c3dab840 x1874671626815360/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828222 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828197.462639:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626815360/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828222 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828197.462648:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828197.462651:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828197.462654:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626815360:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828197.462666:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626815488:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828197.462684:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828197.462694:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828197/real 1787828197]  req@ffff8cd0c3da8000 x1874671626815488/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828222 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828197.462706:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626815488/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828222 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828197.462715:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828197.462718:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828197.462721:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626815488:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828197.654168:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828198.038277:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828198.678160:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828199.062209:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828199.702191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828200.086224:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828200.726240:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828201.110235:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828201.750185:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828202.134261:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828202.582183:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828202.582194:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828202.582199:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828202.582211:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2088
00000100:00080000:3.0:1787828202.582223:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828202.582230:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828202.582273:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626815616/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828202.582303:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828202.582306:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828202.582307:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828202.582328:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2088
00000100:00080000:3.0:1787828202.582331:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828202.582336:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828202.582346:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828202.582357:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626815616:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828202.582362:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626815744/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828202.582385:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828202.582389:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828202.582391:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828202.582398:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2088
00000100:00080000:3.0:1787828202.582403:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828202.582409:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828202.582416:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787828202.582437:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828202.582437:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626815872/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828202.582452:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828202.582455:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828202.582457:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828202.582461:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2088
00000100:00080000:3.0:1787828202.582465:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828202.582469:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828202.582473:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828202.582483:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828202/real 1787828202]  req@ffff8cd0c8eaf0c0 x1874671626815616/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828227 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828202.582485:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626816000/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828202.582501:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626815616/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828227 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828202.582523:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828202.582530:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828202.582533:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626815616:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828202.582572:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626815744:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828202.582592:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828202.582601:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828202/real 1787828202]  req@ffff8cd0c8eac3c0 x1874671626815744/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828227 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828202.582610:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626815744/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828227 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828202.582616:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828202.582619:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828202.582621:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626815744:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828202.582627:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626815872:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828202.582646:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828202.582658:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828202/real 1787828202]  req@ffff8cd0c8eaf840 x1874671626815872/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828227 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828202.582671:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626815872/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828227 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828202.582680:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828202.582683:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828202.582686:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626815872:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828202.582691:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626816000:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828202.582707:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828202.582715:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828202/real 1787828202]  req@ffff8cd0c8eada40 x1874671626816000/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828227 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828202.582729:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626816000/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828227 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828202.582735:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828202.582737:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828202.582739:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626816000:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828202.774206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828203.158258:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828203.798201:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828204.182226:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828204.822208:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828205.206225:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828205.846172:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828206.230213:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828206.870195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828207.254185:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828207.702231:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828207.702241:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828207.702246:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828207.702258:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2093
00000100:00080000:0.0:1787828207.702270:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828207.702276:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828207.702321:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626816128/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828207.702349:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828207.702352:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828207.702353:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828207.702356:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2093
00000100:00080000:0.0:1787828207.702358:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828207.702360:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828207.702370:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828207.702380:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626816256/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828207.702406:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626816128:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828207.702416:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828207.702420:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828207.702421:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828207.702425:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2093
00000100:00080000:0.0:1787828207.702428:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828207.702430:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828207.702434:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828207.702444:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626816384/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828207.702451:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828207.702454:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828207.702455:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828207.702458:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2093
00000100:00080000:0.0:1787828207.702461:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828207.702464:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828207.702466:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828207.702474:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626816512/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787828207.702488:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828207.702542:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828207/real 1787828207]  req@ffff8cd0c8e9d680 x1874671626816128/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828232 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828207.702567:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626816128/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828232 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828207.702576:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828207.702579:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828207.702582:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626816128:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828207.702607:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626816256:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828207.702626:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828207.702635:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828207/real 1787828207]  req@ffff8cd0c8e9f840 x1874671626816256/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828232 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828207.702650:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626816256/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828232 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828207.702656:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828207.702658:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828207.702660:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626816256:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828207.702664:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626816384:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828207.702687:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828207.702697:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828207/real 1787828207]  req@ffff8cd0c8e9ed00 x1874671626816384/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828232 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828207.702707:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626816384/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828232 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828207.702713:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828207.702715:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828207.702718:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626816384:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828207.702723:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626816512:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828207.702738:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828207.702745:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828207/real 1787828207]  req@ffff8cd0c8e9c3c0 x1874671626816512/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828232 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828207.702752:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626816512/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828232 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828207.702766:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828207.702769:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828207.702771:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626816512:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828207.894231:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828208.278273:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828208.918166:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828209.302231:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828209.942189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828210.326227:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828210.966184:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828211.350202:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828211.990191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828212.374259:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828212.822169:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828212.822179:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828212.822206:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828212.822218:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2098
00000100:00080000:0.0:1787828212.822233:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828212.822242:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828212.822286:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626816640/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828212.822315:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828212.822317:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828212.822319:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828212.822321:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2098
00000100:00080000:0.0:1787828212.822324:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828212.822327:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828212.822337:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828212.822347:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626816768/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828212.822359:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828212.822363:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828212.822367:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1787828212.822369:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626816640:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828212.822375:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2098
00000100:00080000:0.0:1787828212.822380:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828212.822385:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828212.822392:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828212.822406:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626816896/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828212.822424:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828212.822427:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828212.822429:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828212.822433:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2098
00000100:00080000:0.0:1787828212.822437:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828212.822441:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:2.0:1787828212.822447:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:0.0:1787828212.822451:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828212.822468:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626817024/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828212.822505:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828212/real 1787828212]  req@ffff8cd0c8e9e940 x1874671626816640/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828237 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828212.822526:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626816640/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828237 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828212.822534:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828212.822538:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828212.822541:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626816640:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828212.822564:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626816768:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828212.822584:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828212.822593:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828212/real 1787828212]  req@ffff8cd0c8e9cf00 x1874671626816768/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828237 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828212.822601:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626816768/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828237 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828212.822613:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828212.822616:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828212.822618:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626816768:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828212.822622:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626816896:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828212.822638:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828212.822645:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828212/real 1787828212]  req@ffff8cd0c8e9fc00 x1874671626816896/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828237 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828212.822652:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626816896/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828237 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828212.822657:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828212.822659:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828212.822661:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626816896:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828212.822665:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626817024:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828212.822675:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828212.822681:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828212/real 1787828212]  req@ffff8cd0c8e9e580 x1874671626817024/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828237 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828212.822689:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626817024/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828237 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828212.822694:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828212.822695:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828212.822702:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626817024:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828213.014179:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828213.398190:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828214.038179:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828214.422262:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828215.062152:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828215.446201:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828216.086162:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828216.470290:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828217.110193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828217.494262:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828217.942165:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828217.942180:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828217.942185:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828217.942196:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2103
00000100:00080000:0.0:1787828217.942208:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828217.942214:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828217.942258:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626817152/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828217.942284:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828217.942287:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828217.942289:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828217.942308:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2103
00000100:00080000:0.0:1787828217.942312:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828217.942316:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828217.942327:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828217.942334:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626817152:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787828217.942347:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626817280/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828217.942361:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828217.942365:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828217.942368:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828217.942373:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2103
00000100:00080000:0.0:1787828217.942379:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828217.942383:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828217.942389:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828217.942406:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626817408/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787828217.942406:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787828217.942421:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828217.942424:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828217.942426:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828217.942430:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2103
00000100:00080000:0.0:1787828217.942433:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828217.942437:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828217.942442:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828217.942452:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828217/real 1787828217]  req@ffff8cd0c8e9de00 x1874671626817152/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828242 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787828217.942454:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626817536/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828217.942470:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626817152/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828242 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828217.942503:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828217.942508:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828217.942523:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626817152:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828217.942557:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626817280:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828217.942580:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828217.942591:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828217/real 1787828217]  req@ffff8cd0c8e9e580 x1874671626817280/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828242 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828217.942599:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626817280/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828242 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828217.942606:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828217.942608:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828217.942616:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626817280:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828217.942621:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626817408:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828217.942637:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828217.942644:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828217/real 1787828217]  req@ffff8cd0c8e9fc00 x1874671626817408/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828242 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828217.942651:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626817408/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828242 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828217.942656:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828217.942658:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828217.942660:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626817408:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828217.942663:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626817536:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828217.942674:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828217.942680:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828217/real 1787828217]  req@ffff8cd0c8e9cf00 x1874671626817536/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828242 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828217.942687:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626817536/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828242 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828217.942691:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828217.942693:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828217.942695:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626817536:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828218.134157:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828218.518328:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828219.158239:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828219.542177:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828220.182174:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828220.566213:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828221.206145:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828221.590166:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828222.230202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828222.614295:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828223.062173:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828223.062187:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828223.062193:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828223.062204:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2108
00000100:00080000:0.0:1787828223.062215:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828223.062222:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828223.062266:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626817664/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828223.062292:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828223.062295:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828223.062297:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828223.062299:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2108
00000100:00080000:0.0:1787828223.062302:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828223.062305:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828223.062314:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828223.062323:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626817792/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828223.062331:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828223.062334:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828223.062335:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828223.062337:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2108
00000100:00080000:0.0:1787828223.062351:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828223.062354:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828223.062359:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828223.062370:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626817920/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828223.062378:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828223.062381:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828223.062383:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828223.062385:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2108
00000100:00080000:0.0:1787828223.062387:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828223.062390:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828223.062393:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828223.062400:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626818048/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828223.062413:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626817664:192.168.203.145@tcp:250:kworker.0
00000800:00000100:2.0:1787828223.062480:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828223.062519:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828223/real 1787828223]  req@ffff8cd0c8e9e940 x1874671626817664/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828248 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828223.062533:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626817664/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828248 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828223.062542:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828223.062545:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828223.062548:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626817664:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828223.062553:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626817792:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828223.062567:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828223.062575:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828223/real 1787828223]  req@ffff8cd0c8e9c3c0 x1874671626817792/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828248 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828223.062582:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626817792/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828248 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828223.062588:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828223.062590:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828223.062592:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626817792:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828223.062597:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626817920:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828223.062627:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828223.062634:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828223/real 1787828223]  req@ffff8cd0c8e9ed00 x1874671626817920/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828248 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828223.062642:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626817920/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828248 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828223.062648:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828223.062650:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828223.062653:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626817920:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828223.062675:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626818048:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828223.062685:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828223.062691:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828223/real 1787828223]  req@ffff8cd0c8e9f840 x1874671626818048/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828248 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828223.062699:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626818048/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828248 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828223.062704:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828223.062706:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828223.062708:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626818048:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828223.254207:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828223.638228:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828224.278202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828224.662291:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828225.302202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828225.686195:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828226.326140:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828226.710251:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828227.350146:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828227.734160:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828228.182194:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828228.182202:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828228.182206:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828228.182214:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2114
00000100:00080000:0.0:1787828228.182225:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828228.182231:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828228.182264:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626818176/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828228.182298:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828228.182301:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828228.182303:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828228.182306:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2114
00000100:00080000:0.0:1787828228.182309:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828228.182312:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828228.182320:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828228.182331:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626818304/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828228.182343:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1787828228.182348:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626818176:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828228.182350:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828228.182353:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828228.182359:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2114
00000100:00080000:0.0:1787828228.182363:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828228.182370:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828228.182377:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828228.182391:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626818432/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828228.182400:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828228.182402:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828228.182404:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828228.182406:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2114
00000100:00080000:0.0:1787828228.182416:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828228.182420:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:2.0:1787828228.182425:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:0.0:1787828228.182428:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828228.182440:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626818560/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828228.182474:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828228/real 1787828228]  req@ffff8cd0c8e9d680 x1874671626818176/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828253 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828228.182492:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626818176/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828253 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828228.182500:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828228.182510:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828228.182513:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626818176:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828228.182533:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626818304:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828228.182553:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828228.182563:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828228/real 1787828228]  req@ffff8cd0c8e9f480 x1874671626818304/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828253 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828228.182572:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626818304/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828253 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828228.182578:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828228.182581:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828228.182583:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626818304:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828228.182588:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626818432:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828228.182600:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828228.182607:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828228/real 1787828228]  req@ffff8cd0c8e9cb40 x1874671626818432/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828253 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828228.182626:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626818432/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828253 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828228.182632:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828228.182634:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828228.182636:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626818432:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828228.182647:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626818560:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828228.182662:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828228.182669:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828228/real 1787828228]  req@ffff8cd0c8e9c000 x1874671626818560/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828253 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828228.182676:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626818560/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828253 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828228.182682:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828228.182684:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828228.182691:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626818560:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828228.374158:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828228.758320:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828229.398178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828229.782217:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828230.422165:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828230.806261:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828231.446187:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828231.830196:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828232.470195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828232.854318:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828233.302285:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828233.302299:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828233.302304:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828233.302341:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2119
00000100:00080000:2.0:1787828233.302356:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828233.302366:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828233.302411:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626818688/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828233.302457:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828233.302465:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828233.302469:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828233.302476:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2119
00000100:00080000:2.0:1787828233.302483:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828233.302490:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828233.302507:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828233.302536:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626818816/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828233.302560:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828233.302567:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828233.302572:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828233.302580:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2119
00000100:00100000:3.0:1787828233.302592:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626818688:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828233.302608:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828233.302617:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828233.302626:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828233.302648:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626818944/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828233.302683:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828233.302689:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828233.302692:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828233.302698:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2119
00000100:00080000:2.0:1787828233.302704:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1787828233.302715:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787828233.302716:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828233.302730:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828233.302749:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626819072/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828233.302801:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828233/real 1787828233]  req@ffff8cd0c3dabc00 x1874671626818688/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828258 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828233.302821:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626818688/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828258 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828233.302832:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828233.302835:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828233.302839:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626818688:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828233.302845:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626818816:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828233.302903:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828233.302914:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828233/real 1787828233]  req@ffff8cd0c3da9e00 x1874671626818816/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828258 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828233.302922:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626818816/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828258 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828233.302928:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828233.302931:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828233.302933:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626818816:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828233.302959:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626818944:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828233.302973:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828233.302980:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828233/real 1787828233]  req@ffff8cd0c3da92c0 x1874671626818944/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828258 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828233.302988:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626818944/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828258 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828233.302993:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828233.302995:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828233.302997:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626818944:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828233.303001:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626819072:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828233.303012:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828233.303018:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828233/real 1787828233]  req@ffff8cd0c3daa1c0 x1874671626819072/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828258 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828233.303036:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626819072/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828258 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828233.303042:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828233.303044:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828233.303046:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626819072:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828233.494147:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828233.878251:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828234.518161:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828234.902269:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828235.542166:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828235.926225:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828236.566157:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828236.950231:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828237.590135:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828237.974240:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828238.422273:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828238.422284:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828238.422288:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828238.422298:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2124
00000100:00080000:0.0:1787828238.422310:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828238.422318:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828238.422370:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d2c0 x1874671626819200/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828238.422401:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828238.422405:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828238.422407:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828238.422410:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2124
00000100:00080000:0.0:1787828238.422414:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828238.422417:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828238.422427:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828238.422461:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626819328/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828238.422470:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626819200:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828238.422480:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828238.422485:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828238.422487:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828238.422491:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2124
00000100:00080000:0.0:1787828238.422496:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828238.422500:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828238.422522:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828238.422537:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626819456/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828238.422547:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828238.422551:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828238.422554:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:3.0:1787828238.422557:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787828238.422563:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2124
00000100:00080000:0.0:1787828238.422569:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828238.422590:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828238.422598:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828238.422614:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626819584/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828238.422641:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828238/real 1787828238]  req@ffff8cd0c8e9d2c0 x1874671626819200/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828263 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828238.422671:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d2c0 x1874671626819200/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828263 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828238.422688:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828238.422707:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828238.422712:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626819200:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828238.422739:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626819328:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828238.422781:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828238.422796:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828238/real 1787828238]  req@ffff8cd0c8e9c000 x1874671626819328/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828263 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828238.422808:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626819328/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828263 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828238.422825:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828238.422829:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828238.422874:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626819328:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828238.422884:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626819456:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828238.422916:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828238.422932:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828238/real 1787828238]  req@ffff8cd0c8e9cb40 x1874671626819456/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828263 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828238.422946:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626819456/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828263 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828238.422957:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828238.422960:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828238.422964:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626819456:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828238.422971:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626819584:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828238.422993:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828238.423012:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828238/real 1787828238]  req@ffff8cd0c8e9f480 x1874671626819584/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828263 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828238.423033:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626819584/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828263 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828238.423049:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828238.423055:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828238.423094:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626819584:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828238.614160:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828238.998264:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828239.638222:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828240.022329:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828240.662210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828241.046309:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828241.686246:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828242.070346:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828242.710183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828243.094334:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828243.542233:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828243.542247:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828243.542253:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828243.542267:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2129
00000100:00080000:1.0:1787828243.542283:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828243.542293:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828243.542351:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626819712/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828243.542391:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828243.542396:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828243.542398:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828243.542402:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2129
00000100:00080000:1.0:1787828243.542406:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828243.542411:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828243.542424:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828243.542440:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626819840/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828243.542454:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828243.542458:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787828243.542461:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828243.542465:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2129
00000100:00080000:1.0:1787828243.542468:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1787828243.542482:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626819712:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787828243.542501:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828243.542509:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828243.542527:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626819968/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828243.542545:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828243.542552:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828243.542555:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828243.542563:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2129
00000100:00080000:1.0:1787828243.542571:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828243.542579:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:3.0:1787828243.542587:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:1.0:1787828243.542591:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828243.542618:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626820096/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828243.542688:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828243/real 1787828243]  req@ffff8cd0d48f7840 x1874671626819712/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828268 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828243.542717:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626819712/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828268 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828243.542733:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828243.542738:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828243.542743:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626819712:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828243.542754:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626819840:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828243.542786:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828243.542832:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828243/real 1787828243]  req@ffff8cd0d48f5a40 x1874671626819840/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828268 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828243.542875:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626819840/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828268 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828243.542890:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828243.542896:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828243.542903:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626819840:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828243.542968:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626819968:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828243.543016:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828243.543032:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828243/real 1787828243]  req@ffff8cd0d48f43c0 x1874671626819968/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828268 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828243.543046:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626819968/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828268 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828243.543056:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828243.543060:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828243.543064:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626819968:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828243.543071:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626820096:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828243.543139:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828243.543154:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828243/real 1787828243]  req@ffff8cd0d48f70c0 x1874671626820096/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828268 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828243.543168:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626820096/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828268 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828243.543178:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828243.543181:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828243.543185:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626820096:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828243.734248:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828244.118295:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828244.758190:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828245.142347:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828245.782200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828246.166357:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828246.806227:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828247.190416:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828247.830292:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828248.214210:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828248.662212:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828248.662220:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828248.662224:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828248.662233:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2134
00000100:00080000:0.0:1787828248.662243:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828248.662249:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828248.662288:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626820224/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828248.662310:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828248.662314:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828248.662315:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828248.662318:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2134
00000100:00080000:0.0:1787828248.662320:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828248.662323:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828248.662331:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828248.662339:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626820352/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828248.662348:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828248.662353:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:3.0:1787828248.662354:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626820224:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828248.662355:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828248.662362:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2134
00000100:00080000:0.0:1787828248.662364:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828248.662366:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828248.662382:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828248.662391:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626820480/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828248.662398:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828248.662401:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828248.662403:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828248.662406:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2134
00000100:00080000:0.0:1787828248.662410:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828248.662413:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828248.662417:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828248.662424:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626820608/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828248.662474:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626820352:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828248.662507:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626820480:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828248.662524:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626820608:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828248.854138:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828249.238313:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828249.878213:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828250.262276:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828250.902149:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828251.286223:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:1.0:1787828251.734344:0:2344:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:1.0:1787828251.734386:0:2344:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1787828251.734403:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787828251.734439:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787828251.734449:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787828251.734460:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:3.0:1787828251.734509:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828248/real 1787828251]  req@ffff8cd0c8e9f840 x1874671626820352/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828273 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828251.734550:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626820352/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828273 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828251.734565:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828251.734571:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828251.734577:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626820352:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828251.734587:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828248/real 1787828251]  req@ffff8cd0c8e9ed00 x1874671626820480/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828273 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828251.734602:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626820480/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828273 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828251.734608:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828251.734610:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828251.734613:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626820480:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828251.734618:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828248/real 1787828251]  req@ffff8cd0c8e9c3c0 x1874671626820608/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828273 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828251.734659:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626820608/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828273 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828251.734668:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828251.734671:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828251.734674:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626820608:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828251.734681:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828248/real 1787828251]  req@ffff8cd0c8e9d680 x1874671626820224/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828273 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828251.734694:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626820224/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828273 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828251.734702:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828251.734705:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828251.734708:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626820224:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828251.926154:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828252.310308:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828252.950193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828253.334228:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828253.782329:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828253.782340:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828253.782347:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828253.782370:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2139
00000100:00080000:0.0:1787828253.782389:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828253.782404:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828253.782462:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626820736/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828253.782495:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828253.782499:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828253.782518:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828253.782524:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2139
00000100:00080000:0.0:1787828253.782530:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828253.782536:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828253.782549:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828253.782552:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626820736:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787828253.782568:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626820864/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828253.782595:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828253.782599:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828253.782602:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828253.782606:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2139
00000100:00080000:0.0:1787828253.782610:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828253.782615:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828253.782619:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828253.782631:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626820992/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828253.782643:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:3.0:1787828253.782644:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787828253.782652:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828253.782655:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828253.782662:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2139
00000100:00080000:0.0:1787828253.782666:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828253.782671:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828253.782677:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828253.782698:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626821120/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828253.782705:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828253/real 1787828253]  req@ffff8cd0c8e9e940 x1874671626820736/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828278 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828253.782758:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626820736/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828278 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828253.782773:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828253.782778:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828253.782792:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626820736:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828253.782821:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626820864:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828253.782849:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828253.782864:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828253/real 1787828253]  req@ffff8cd0c8e9cf00 x1874671626820864/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828278 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828253.782880:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626820864/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828278 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828253.782894:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828253.782900:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828253.782913:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626820864:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828253.782925:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626820992:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828253.782957:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828253.782976:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828253/real 1787828253]  req@ffff8cd0c8e9fc00 x1874671626820992/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828278 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828253.782995:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626820992/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828278 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828253.783007:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828253.783012:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828253.783017:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626820992:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828253.783026:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626821120:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828253.783055:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828253.783136:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828253/real 1787828253]  req@ffff8cd0c8e9e580 x1874671626821120/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828278 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828253.783160:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626821120/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828278 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828253.783172:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828253.783177:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828253.783182:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626821120:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828253.974168:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828254.358161:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828254.998178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828255.382358:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828256.022221:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828256.406227:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828257.046177:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828257.430285:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828258.070133:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828258.454177:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828258.902185:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828258.902193:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828258.902197:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828258.902205:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2144
00000100:00080000:2.0:1787828258.902214:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828258.902220:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828258.902252:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daad00 x1874671626821248/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828258.902275:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828258.902278:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828258.902279:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828258.902282:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2144
00000100:00080000:2.0:1787828258.902284:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828258.902287:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828258.902295:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828258.902308:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626821376/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828258.902319:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828258.902323:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828258.902340:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828258.902361:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2144
00000100:00100000:3.0:1787828258.902372:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626821248:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828258.902392:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828258.902405:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828258.902415:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828258.902436:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626821504/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828258.902469:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828258.902473:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828258.902479:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:3.0:1787828258.902481:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787828258.902488:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2144
00000100:00080000:2.0:1787828258.902496:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828258.902502:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828258.902509:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828258.902524:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626821632/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828258.902575:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828258/real 1787828258]  req@ffff8cd0c3daad00 x1874671626821248/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828283 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828258.902596:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daad00 x1874671626821248/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828283 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828258.902607:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828258.902610:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828258.902613:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626821248:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828258.902637:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626821376:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828258.902670:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828258.902686:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828258/real 1787828258]  req@ffff8cd0c3daa1c0 x1874671626821376/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828283 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828258.902755:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626821376/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828283 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828258.902761:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828258.902763:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828258.902766:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626821376:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828258.902784:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626821504:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828258.902800:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828258.902808:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828258/real 1787828258]  req@ffff8cd0c3da92c0 x1874671626821504/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828283 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828258.902816:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626821504/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828283 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828258.902821:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828258.902824:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828258.902826:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626821504:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828258.902830:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626821632:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828258.902839:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828258.902845:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828258/real 1787828258]  req@ffff8cd0c3da9e00 x1874671626821632/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828283 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828258.902852:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626821632/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828283 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828258.902867:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828258.902869:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828258.902871:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626821632:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828259.094154:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828259.478343:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828260.118172:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828260.502307:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828261.142201:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828261.526302:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828262.166187:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828262.550196:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828263.190268:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828263.574281:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828264.022236:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828264.022249:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828264.022256:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828264.022280:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2149
00000100:00080000:0.0:1787828264.022296:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828264.022305:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828264.022354:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626821760/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828264.022391:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828264.022396:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828264.022398:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828264.022403:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2149
00000100:00080000:0.0:1787828264.022408:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828264.022413:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828264.022426:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828264.022461:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626821888/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828264.022471:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626821760:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828264.022507:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828264.022535:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828264.022540:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828264.022551:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2149
00000100:00080000:0.0:1787828264.022562:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828264.022569:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828264.022579:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787828264.022580:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787828264.022606:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626822016/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828264.022623:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828264.022628:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828264.022631:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828264.022638:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2149
00000100:00080000:0.0:1787828264.022645:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828264.022651:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828264.022682:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828264/real 1787828264]  req@ffff8cd0c8e9de00 x1874671626821760/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828289 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:0.0:1787828264.022687:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828264.022715:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626822144/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828264.022727:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626821760/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828289 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828264.022755:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828264.022761:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828264.022768:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626821760:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828264.022807:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626821888:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828264.022844:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828264.022862:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828264/real 1787828264]  req@ffff8cd0c8e9e580 x1874671626821888/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828289 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828264.022878:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626821888/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828289 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828264.022909:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828264.022914:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828264.022933:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626821888:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828264.022943:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626822016:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828264.022976:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828264.022991:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828264/real 1787828264]  req@ffff8cd0c8e9fc00 x1874671626822016/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828289 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828264.023006:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626822016/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828289 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828264.023017:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828264.023021:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828264.023026:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626822016:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828264.023033:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626822144:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828264.023066:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828264.023079:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828264/real 1787828264]  req@ffff8cd0c8e9cf00 x1874671626822144/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828289 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828264.023144:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626822144/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828289 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828264.023160:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828264.023166:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828264.023186:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626822144:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828264.214210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828264.598275:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828265.238209:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828265.622364:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828266.262234:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828266.646202:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828267.286265:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828267.670293:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828268.310210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828268.694241:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828269.142201:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828269.142213:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828269.142218:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828269.142230:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2154
00000100:00080000:0.0:1787828269.142295:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828269.142305:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828269.142351:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626822272/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828269.142414:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828269.142421:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828269.142425:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828269.142432:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2154
00000100:00080000:0.0:1787828269.142439:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828269.142447:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828269.142464:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828269.142506:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626822400/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828269.142516:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626822272:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828269.142536:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828269.142542:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828269.142545:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828269.142553:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2154
00000100:00080000:0.0:1787828269.142559:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828269.142564:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828269.142572:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828269.142590:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626822528/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828269.142604:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:3.0:1787828269.142611:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787828269.142684:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828269.142688:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828269.142697:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2154
00000100:00080000:0.0:1787828269.142706:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1787828269.142708:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828269/real 1787828269]  req@ffff8cd0c8e9e940 x1874671626822272/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828294 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787828269.142718:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828269.142737:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828269.142771:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626822272/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828294 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787828269.142786:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626822656/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828269.142820:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828269.142830:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828269.142837:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626822272:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828269.142870:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626822400:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828269.142922:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828269.142939:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828269/real 1787828269]  req@ffff8cd0c8e9d680 x1874671626822400/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828294 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828269.142957:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626822400/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828294 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828269.142975:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828269.142979:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828269.142983:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626822400:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828269.142991:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626822528:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828269.143049:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828269.143083:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828269/real 1787828269]  req@ffff8cd0c8e9c3c0 x1874671626822528/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828294 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828269.143167:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626822528/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828294 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828269.143184:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828269.143202:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828269.143207:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626822528:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828269.143216:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626822656:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828269.143246:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828269.143271:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828269/real 1787828269]  req@ffff8cd0c8e9ed00 x1874671626822656/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828294 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828269.143285:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626822656/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828294 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828269.143313:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828269.143317:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828269.143321:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626822656:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828269.334256:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828269.718308:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828270.358229:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828270.742179:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828271.382251:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828271.766352:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828272.406174:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828272.790339:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828273.430207:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828273.814283:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828274.262190:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828274.262204:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828274.262210:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828274.262222:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2160
00000100:00080000:0.0:1787828274.262237:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828274.262248:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828274.262299:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626822784/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828274.262345:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828274.262351:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828274.262353:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828274.262358:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2160
00000100:00080000:0.0:1787828274.262363:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828274.262368:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828274.262409:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828274.262438:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626822912/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828274.262464:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828274.262472:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828274.262476:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1787828274.262486:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626822784:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828274.262489:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2160
00000100:00080000:0.0:1787828274.262502:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828274.262511:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828274.262520:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828274.262541:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626823040/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828274.262557:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828274.262563:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828274.262566:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828274.262572:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2160
00000100:00080000:0.0:1787828274.262577:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828274.262609:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:3.0:1787828274.262620:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:0.0:1787828274.262622:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828274.262645:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626823168/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828274.262705:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828274/real 1787828274]  req@ffff8cd0c8e9f840 x1874671626822784/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828299 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828274.262732:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626822784/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828299 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828274.262747:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828274.262752:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828274.262758:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626822784:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828274.262768:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626822912:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828274.262800:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828274.262815:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828274/real 1787828274]  req@ffff8cd0c8e9f480 x1874671626822912/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828299 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828274.262831:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626822912/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828299 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828274.262841:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828274.262845:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828274.262851:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626822912:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828274.262891:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626823040:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828274.262932:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828274.262956:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828274/real 1787828274]  req@ffff8cd0c8e9cb40 x1874671626823040/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828299 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828274.262982:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626823040/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828299 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828274.263029:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828274.263034:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828274.263039:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626823040:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828274.263048:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626823168:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828274.263079:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828274.263138:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828274/real 1787828274]  req@ffff8cd0c8e9c000 x1874671626823168/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828299 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828274.263153:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626823168/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828299 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828274.263164:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828274.263168:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828274.263172:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626823168:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828274.454177:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828274.838224:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828275.478162:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828275.862294:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828276.502133:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828276.886214:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828277.526196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828277.910298:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828278.550215:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828278.934251:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828279.382206:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828279.382216:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828279.382221:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828279.382232:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2165
00000100:00080000:0.0:1787828279.382243:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828279.382251:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828279.382289:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d2c0 x1874671626823296/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828279.382319:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828279.382323:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828279.382326:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828279.382330:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2165
00000100:00080000:0.0:1787828279.382333:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828279.382338:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828279.382348:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828279.382365:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626823424/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828279.382371:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626823296:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828279.382386:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828279.382390:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828279.382392:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828279.382397:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2165
00000100:00080000:0.0:1787828279.382402:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828279.382406:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828279.382428:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787828279.382434:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787828279.382448:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626823552/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828279.382461:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828279.382465:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828279.382467:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1787828279.382472:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828279/real 1787828279]  req@ffff8cd0c8e9d2c0 x1874671626823296/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828304 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787828279.382474:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2165
00000100:00080000:0.0:1787828279.382484:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828279.382491:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828279.382494:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d2c0 x1874671626823296/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828304 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:0.0:1787828279.382501:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00080000:3.0:1787828279.382505:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:0.0:1787828279.382519:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626823680/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828279.382522:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828279.382529:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626823296:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828279.382570:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626823424:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828279.382593:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828279.382604:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828279/real 1787828279]  req@ffff8cd0c8e9c000 x1874671626823424/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828304 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828279.382612:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626823424/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828304 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828279.382618:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828279.382620:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828279.382623:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626823424:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828279.382627:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626823552:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828279.382639:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828279.382645:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828279/real 1787828279]  req@ffff8cd0c8e9cb40 x1874671626823552/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828304 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828279.382653:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626823552/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828304 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828279.382658:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828279.382659:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828279.382661:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626823552:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828279.382665:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626823680:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828279.382683:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828279.382691:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828279/real 1787828279]  req@ffff8cd0c8e9f480 x1874671626823680/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828304 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828279.382698:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626823680/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828304 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828279.382703:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828279.382705:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828279.382707:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626823680:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828279.574190:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828279.958240:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828280.598146:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828280.982202:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828281.622187:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828282.006265:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828282.646136:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828283.030291:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828283.670143:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828284.054260:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828284.502213:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828284.502220:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828284.502224:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828284.502233:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2170
00000100:00080000:1.0:1787828284.502242:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828284.502266:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828284.502296:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626823808/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828284.502319:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828284.502321:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828284.502323:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828284.502325:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2170
00000100:00080000:1.0:1787828284.502329:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828284.502331:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828284.502339:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828284.502347:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626823936/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828284.502357:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828284.502360:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787828284.502361:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828284.502363:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2170
00000100:00080000:1.0:1787828284.502365:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828284.502367:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828284.502370:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828284.502378:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626824064/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828284.502389:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787828284.502389:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626823808:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787828284.502402:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828284.502409:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828284.502413:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2170
00000100:00080000:1.0:1787828284.502417:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828284.502421:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828284.502426:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828284.502438:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626824192/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828284.502481:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828284.502540:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828284/real 1787828284]  req@ffff8cd0d48f61c0 x1874671626823808/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828309 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828284.502557:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626823808/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828309 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828284.502566:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828284.502570:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828284.502572:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626823808:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828284.502579:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626823936:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828284.502597:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828284.502617:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828284/real 1787828284]  req@ffff8cd0d48f7c00 x1874671626823936/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828309 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828284.502626:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626823936/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828309 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828284.502632:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828284.502634:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828284.502636:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626823936:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828284.502662:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626824064:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828284.502676:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828284.502683:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828284/real 1787828284]  req@ffff8cd0d48f52c0 x1874671626824064/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828309 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828284.502691:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626824064/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828309 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828284.502696:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828284.502698:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828284.502699:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626824064:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828284.502703:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626824192:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828284.502714:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828284.502719:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828284/real 1787828284]  req@ffff8cd0d48f6d00 x1874671626824192/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828309 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828284.502727:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626824192/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828309 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828284.502736:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828284.502739:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828284.502741:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626824192:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828284.694191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828285.078226:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828285.718183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828286.102198:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828286.742207:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828287.126183:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828287.766169:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828288.150333:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828288.790195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828289.174239:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828289.622185:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828289.622193:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828289.622197:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828289.622205:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2175
00000100:00080000:3.0:1787828289.622213:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828289.622219:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828289.622251:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626824320/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828289.622281:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828289.622284:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828289.622286:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828289.622288:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2175
00000100:00080000:3.0:1787828289.622291:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828289.622293:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828289.622302:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828289.622310:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626824448/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828289.622319:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1787828289.622323:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626824320:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787828289.622337:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828289.622338:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828289.622343:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2175
00000100:00080000:3.0:1787828289.622346:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828289.622349:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828289.622353:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828289.622363:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626824576/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828289.622371:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828289.622374:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828289.622375:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828289.622377:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2175
00000100:00080000:3.0:1787828289.622380:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1787828289.622383:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787828289.622386:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828289.622391:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828289.622399:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626824704/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828289.622426:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828289/real 1787828289]  req@ffff8cd0c8eac780 x1874671626824320/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828314 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828289.622441:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626824320/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828314 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828289.622450:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828289.622453:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828289.622456:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626824320:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828289.622492:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626824448:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828289.622511:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828289.622520:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828289/real 1787828289]  req@ffff8cd0c8eae1c0 x1874671626824448/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828314 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828289.622529:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626824448/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828314 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828289.622535:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828289.622537:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828289.622540:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626824448:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828289.622543:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626824576:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828289.622555:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828289.622561:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828289/real 1787828289]  req@ffff8cd0c8ead2c0 x1874671626824576/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828314 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828289.622569:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626824576/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828314 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828289.622574:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828289.622576:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828289.622578:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626824576:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828289.622582:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626824704:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828289.622592:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828289.622598:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828289/real 1787828289]  req@ffff8cd0c8eae940 x1874671626824704/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828314 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828289.622611:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626824704/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828314 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828289.622617:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828289.622619:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828289.622621:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626824704:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828289.814167:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828290.198240:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828290.838166:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828291.222295:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828291.862182:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828292.246238:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828292.886180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828293.270199:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828293.910178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828294.294338:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828294.742198:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828294.742208:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828294.742213:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828294.742223:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2180
00000100:00080000:3.0:1787828294.742258:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828294.742265:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828294.742304:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626824832/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828294.742331:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828294.742334:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828294.742335:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828294.742337:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2180
00000100:00080000:3.0:1787828294.742340:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828294.742342:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828294.742352:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828294.742361:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626824960/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828294.742370:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828294.742373:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828294.742374:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828294.742378:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2180
00000100:00080000:3.0:1787828294.742381:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1787828294.742384:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626824832:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787828294.742387:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828294.742396:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828294.742406:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626825088/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828294.742413:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828294.742420:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828294.742422:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828294.742425:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2180
00000100:00080000:3.0:1787828294.742429:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828294.742433:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828294.742437:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828294.742459:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626825216/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787828294.742480:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828294.742531:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828294/real 1787828294]  req@ffff8cd0c8eac000 x1874671626824832/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828319 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828294.742550:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626824832/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828319 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828294.742559:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828294.742563:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828294.742566:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626824832:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828294.742589:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626824960:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828294.742607:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828294.742616:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828294/real 1787828294]  req@ffff8cd0c8eacf00 x1874671626824960/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828319 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828294.742632:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626824960/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828319 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828294.742638:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828294.742641:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828294.742644:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626824960:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828294.742651:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626825088:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828294.742674:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828294.742687:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828294/real 1787828294]  req@ffff8cd0c8eae940 x1874671626825088/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828319 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828294.742700:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626825088/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828319 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828294.742708:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828294.742712:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828294.742715:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626825088:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828294.742722:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626825216:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828294.742745:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828294.742758:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828294/real 1787828294]  req@ffff8cd0c8ead2c0 x1874671626825216/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828319 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828294.742767:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626825216/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828319 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828294.742781:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828294.742784:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828294.742786:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626825216:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828294.934216:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828295.318181:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828295.958183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828296.342295:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828296.982258:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828297.366206:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828298.006154:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828298.390275:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828299.030204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828299.414173:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828299.862186:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828299.862203:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828299.862207:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828299.862218:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2185
00000100:00080000:0.0:1787828299.862255:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828299.862262:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828299.862305:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626825344/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828299.862363:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828299.862366:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828299.862368:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828299.862371:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2185
00000100:00080000:0.0:1787828299.862374:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828299.862377:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828299.862392:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828299.862403:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626825472/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828299.862426:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828299.862429:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828299.862431:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828299.862434:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2185
00000100:00080000:0.0:1787828299.862437:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828299.862439:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828299.862444:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828299.862454:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626825600/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828299.862456:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626825344:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828299.862481:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828299.862491:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828299.862493:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828299.862498:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2185
00000100:00080000:0.0:1787828299.862502:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828299.862506:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828299.862510:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828299.862524:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626825728/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787828299.862554:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828299.862594:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828299/real 1787828299]  req@ffff8cd0c8e9f840 x1874671626825344/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828324 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828299.862607:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626825344/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828324 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828299.862617:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828299.862622:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828299.862628:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626825344:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828299.862636:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626825472:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828299.862661:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828299.862676:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828299/real 1787828299]  req@ffff8cd0c8e9ed00 x1874671626825472/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828324 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828299.862687:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626825472/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828324 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828299.862706:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828299.862709:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828299.862711:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626825472:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828299.862734:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626825600:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828299.862751:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828299.862759:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828299/real 1787828299]  req@ffff8cd0c8e9c3c0 x1874671626825600/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828324 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828299.862767:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626825600/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828324 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828299.862772:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828299.862774:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828299.862776:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626825600:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828299.862780:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626825728:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828299.862791:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828299.862797:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828299/real 1787828299]  req@ffff8cd0c8e9d680 x1874671626825728/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828324 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828299.862805:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626825728/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828324 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828299.862809:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828299.862811:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828299.862819:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626825728:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828300.054151:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828300.438288:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828301.078147:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828301.462207:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828302.102159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828302.486201:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828303.126159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828303.510187:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828304.150137:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828304.534236:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828304.982210:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828304.982224:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828304.982230:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828304.982244:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2190
00000100:00080000:1.0:1787828304.982259:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828304.982269:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828304.982320:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626825856/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828304.982352:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828304.982356:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828304.982359:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828304.982364:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2190
00000100:00080000:1.0:1787828304.982382:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828304.982387:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828304.982395:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626825856:192.168.203.145@tcp:250:kworker.0
00000008:00100000:1.0:1787828304.982408:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828304.982431:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626825984/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828304.982471:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828304.982477:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000800:00000100:2.0:1787828304.982478:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787828304.982482:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828304.982488:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2190
00000100:00080000:1.0:1787828304.982494:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828304.982499:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828304.982506:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828304.982521:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626826112/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828304.982529:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828304/real 1787828304]  req@ffff8cd0d48f6d00 x1874671626825856/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828329 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787828304.982538:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828304.982544:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828304.982546:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1787828304.982551:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626825856/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828329 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787828304.982553:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2190
00000100:00080000:1.0:1787828304.982561:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828304.982566:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787828304.982567:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00080000:2.0:1787828304.982572:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828304.982576:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626825856:192.168.203.145@tcp:250:kworker.0
00000008:00100000:1.0:1787828304.982577:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828304.982595:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626826240/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828304.982611:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626825984:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828304.982642:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828304.982656:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828304/real 1787828304]  req@ffff8cd0d48f52c0 x1874671626825984/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828329 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828304.982671:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626825984/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828329 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828304.982680:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828304.982683:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828304.982692:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626825984:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828304.982697:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626826112:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828304.982720:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828304.982732:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828304/real 1787828304]  req@ffff8cd0d48f7c00 x1874671626826112/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828329 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828304.982744:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626826112/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828329 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828304.982754:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828304.982757:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828304.982760:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626826112:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828304.982778:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626826240:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828304.982801:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828304.982814:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828304/real 1787828304]  req@ffff8cd0d48f61c0 x1874671626826240/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828329 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828304.982824:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626826240/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828329 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828304.982830:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828304.982832:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828304.982834:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626826240:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828305.174143:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828305.558177:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828306.198164:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828306.582276:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828307.222193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828307.606203:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828308.246291:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828308.630284:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828309.270190:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828309.654263:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828310.102390:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828310.102399:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828310.102403:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828310.102427:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2195
00000100:00080000:0.0:1787828310.102438:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828310.102445:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828310.102484:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626826368/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828310.102540:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828310.102545:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828310.102547:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828310.102552:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2195
00000100:00080000:0.0:1787828310.102591:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828310.102596:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828310.102607:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828310.102608:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626826368:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787828310.102620:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626826496/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828310.102632:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828310.102636:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828310.102638:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828310.102642:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2195
00000100:00080000:0.0:1787828310.102645:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828310.102649:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828310.102654:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828310.102664:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626826624/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787828310.102672:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787828310.102674:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828310.102677:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828310.102679:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828310.102682:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2195
00000100:00080000:0.0:1787828310.102686:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828310.102690:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828310.102695:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828310.102705:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626826752/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828310.102723:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828310/real 1787828310]  req@ffff8cd0c8e9e940 x1874671626826368/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828335 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828310.102740:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626826368/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828335 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828310.102751:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828310.102771:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828310.102775:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626826368:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828310.102800:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626826496:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828310.102838:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828310.102849:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828310/real 1787828310]  req@ffff8cd0c8e9cf00 x1874671626826496/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828335 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828310.102859:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626826496/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828335 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828310.102865:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828310.102868:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828310.102871:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626826496:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828310.102875:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626826624:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828310.102919:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828310.102927:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828310/real 1787828310]  req@ffff8cd0c8e9fc00 x1874671626826624/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828335 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828310.102937:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626826624/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828335 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828310.102943:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828310.102946:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828310.102948:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626826624:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828310.102953:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626826752:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828310.102965:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828310.102972:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828310/real 1787828310]  req@ffff8cd0c8e9e580 x1874671626826752/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828335 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828310.102980:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626826752/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828335 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828310.102986:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828310.102988:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828310.102991:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626826752:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828310.294203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828310.678268:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828311.318156:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828311.702174:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828312.342139:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828312.726224:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828313.366213:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828313.750209:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828314.390184:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828314.774241:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828315.222196:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828315.222205:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828315.222210:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828315.222219:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2201
00000100:00080000:0.0:1787828315.222231:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828315.222238:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828315.222276:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626826880/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828315.222323:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828315.222328:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828315.222330:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828315.222335:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2201
00000100:00080000:0.0:1787828315.222339:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828315.222343:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828315.222354:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828315.222369:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626827008/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828315.222384:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828315.222392:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828315.222395:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1787828315.222395:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626826880:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828315.222402:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2201
00000100:00080000:0.0:1787828315.222432:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828315.222439:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828315.222445:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828315.222461:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626827136/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828315.222470:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828315.222474:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828315.222476:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828315.222484:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2201
00000100:00080000:0.0:1787828315.222487:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828315.222489:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828315.222494:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828315.222503:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626827264/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828315.222566:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626827008:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828315.222613:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626827136:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828315.222637:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626827264:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828315.414213:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828315.798328:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828316.438259:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828316.822397:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828317.462215:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828317.846326:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:1.0:1787828318.294499:0:2346:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:1.0:1787828318.294551:0:2346:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1787828318.294581:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787828318.294644:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787828318.294687:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787828318.294702:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:2.0:1787828318.294755:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828315/real 1787828318]  req@ffff8cd0c8e9e580 x1874671626827008/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828340 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828318.294819:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626827008/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828340 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828318.294841:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828318.294849:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828318.294856:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626827008:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828318.294870:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828315/real 1787828318]  req@ffff8cd0c8e9fc00 x1874671626827136/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828340 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828318.294893:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626827136/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828340 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828318.294908:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828318.294913:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828318.294919:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626827136:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828318.294930:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828315/real 1787828318]  req@ffff8cd0c8e9cf00 x1874671626827264/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828340 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828318.294969:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626827264/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828340 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828318.294982:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828318.294988:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828318.294993:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626827264:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828318.295003:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828315/real 1787828318]  req@ffff8cd0c8e9de00 x1874671626826880/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828340 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828318.295022:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626826880/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828340 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828318.295035:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828318.295040:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828318.295046:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626826880:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828318.486236:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828318.870283:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828319.510169:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828319.894229:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828320.342377:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828320.342393:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828320.342399:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828320.342411:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2206
00000100:00080000:1.0:1787828320.342425:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828320.342433:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828320.342476:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626827392/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828320.342528:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828320.342533:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828320.342535:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828320.342539:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2206
00000100:00080000:1.0:1787828320.342543:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828320.342548:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828320.342561:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828320.342578:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626827520/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828320.342594:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828320.342603:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:2.0:1787828320.342605:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626827392:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787828320.342607:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828320.342624:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2206
00000100:00080000:1.0:1787828320.342628:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828320.342652:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828320.342662:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828320.342692:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626827648/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828320.342711:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828320.342718:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828320.342722:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1787828320.342733:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787828320.342734:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2206
00000100:00080000:1.0:1787828320.342762:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828320.342773:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828320.342783:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828320.342798:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828320/real 1787828320]  req@ffff8cd0d48f70c0 x1874671626827392/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828345 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787828320.342812:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626827776/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828320.342828:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626827392/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828345 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828320.342858:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828320.342889:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828320.342894:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626827392:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828320.342931:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626827520:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828320.342970:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828320.342996:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828320/real 1787828320]  req@ffff8cd0d48f43c0 x1874671626827520/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828345 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828320.343011:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626827520/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828345 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828320.343020:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828320.343024:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828320.343028:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626827520:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828320.343036:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626827648:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828320.343060:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828320.343074:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828320/real 1787828320]  req@ffff8cd0d48f5a40 x1874671626827648/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828345 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828320.343131:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626827648/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828345 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828320.343141:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828320.343145:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828320.343148:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626827648:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828320.343155:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626827776:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828320.343188:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828320.343201:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828320/real 1787828320]  req@ffff8cd0d48f7840 x1874671626827776/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828345 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828320.343212:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626827776/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828345 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828320.343219:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828320.343223:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828320.343225:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626827776:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828320.534197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828320.918361:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828321.558345:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828321.942208:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828322.582184:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828322.966338:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828323.606266:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828323.990228:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828324.630272:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828325.014275:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828325.462161:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828325.462171:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828325.462176:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828325.462187:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2211
00000100:00080000:1.0:1787828325.462199:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828325.462205:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828325.462265:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626827904/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828325.462292:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828325.462295:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828325.462314:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828325.462317:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2211
00000100:00080000:1.0:1787828325.462320:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828325.462324:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828325.462334:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828325.462345:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626828032/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828325.462353:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828325.462357:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787828325.462358:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828325.462360:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2211
00000100:00080000:1.0:1787828325.462362:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828325.462365:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828325.462369:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828325.462379:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626828160/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828325.462385:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626827904:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787828325.462399:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828325.462402:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828325.462404:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828325.462407:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2211
00000100:00080000:1.0:1787828325.462409:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828325.462412:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828325.462416:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828325.462432:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626828288/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787828325.462466:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828325.462505:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828325/real 1787828325]  req@ffff8cd0d48f4b40 x1874671626827904/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828350 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828325.462518:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626827904/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828350 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828325.462527:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828325.462530:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828325.462533:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626827904:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828325.462538:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626828032:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828325.462569:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828325.462578:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828325/real 1787828325]  req@ffff8cd0d48f6580 x1874671626828032/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828350 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828325.462586:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626828032/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828350 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828325.462592:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828325.462594:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828325.462596:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626828032:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828325.462617:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626828160:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828325.462634:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828325.462645:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828325/real 1787828325]  req@ffff8cd0d48f7480 x1874671626828160/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828350 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828325.462657:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626828160/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828350 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828325.462666:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828325.462670:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828325.462674:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626828160:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828325.462681:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626828288:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828325.462701:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828325.462709:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828325/real 1787828325]  req@ffff8cd0d48f4000 x1874671626828288/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828350 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828325.462728:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626828288/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828350 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828325.462734:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828325.462736:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828325.462738:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626828288:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828325.654210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828326.038276:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828326.678179:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828327.062197:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828327.702343:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828328.086168:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828328.726182:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828329.110283:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828329.750254:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828330.134268:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828330.582344:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828330.582359:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828330.582365:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828330.582378:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2216
00000100:00080000:0.0:1787828330.582412:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828330.582424:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828330.582470:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626828416/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828330.582522:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828330.582527:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828330.582530:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828330.582536:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2216
00000100:00080000:0.0:1787828330.582541:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828330.582547:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828330.582560:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828330.582580:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626828544/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828330.582590:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626828416:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828330.582606:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828330.582612:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828330.582615:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828330.582622:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2216
00000100:00080000:0.0:1787828330.582628:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828330.582634:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828330.582643:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828330.582659:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626828672/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787828330.582662:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787828330.582697:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828330.582712:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828330.582716:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828330.582725:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2216
00000100:00100000:2.0:1787828330.582731:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828330/real 1787828330]  req@ffff8cd0c8e9e940 x1874671626828416/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828355 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787828330.582735:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828330.582750:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828330.582760:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828330.582761:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626828416/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828355 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828330.582779:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828330.582786:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787828330.582788:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626828800/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828330.582793:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626828416:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828330.582829:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626828544:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828330.582855:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828330.582868:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828330/real 1787828330]  req@ffff8cd0c8e9d680 x1874671626828544/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828355 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828330.582910:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626828544/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828355 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828330.582918:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828330.582921:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828330.582924:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626828544:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828330.582929:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626828672:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828330.582948:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828330.582957:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828330/real 1787828330]  req@ffff8cd0c8e9c3c0 x1874671626828672/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828355 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828330.582989:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626828672/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828355 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828330.583001:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828330.583006:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828330.583011:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626828672:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828330.583021:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626828800:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828330.583066:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828330.583085:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828330/real 1787828330]  req@ffff8cd0c8e9ed00 x1874671626828800/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828355 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828330.583099:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626828800/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828355 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828330.583142:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828330.583147:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828330.583151:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626828800:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828330.774209:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828331.158269:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828331.798175:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828332.182223:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828332.822195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828333.206312:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828333.846157:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828334.230237:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828334.870197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828335.254290:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828335.702286:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828335.702298:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828335.702304:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828335.702316:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2221
00000100:00080000:2.0:1787828335.702329:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828335.702338:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828335.702385:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626828928/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828335.702424:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828335.702428:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828335.702431:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828335.702453:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2221
00000100:00080000:2.0:1787828335.702457:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828335.702461:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828335.702474:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828335.702490:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626829056/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828335.702546:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626828928:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828335.702563:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828335.702579:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828335.702588:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828335.702599:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2221
00000100:00080000:2.0:1787828335.702610:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828335.702619:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828335.702633:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828335.702659:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626829184/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828335.702674:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:3.0:1787828335.702683:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787828335.702685:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828335.702692:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828335.702716:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2221
00000100:00080000:2.0:1787828335.702725:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828335.702732:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828335.702739:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828335.702755:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626829312/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828335.702787:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828335/real 1787828335]  req@ffff8cd0c3dabc00 x1874671626828928/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828360 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828335.702818:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626828928/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828360 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828335.702834:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828335.702839:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828335.702844:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626828928:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828335.702895:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626829056:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828335.702945:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828335.702976:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828335/real 1787828335]  req@ffff8cd0c3da8000 x1874671626829056/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828360 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828335.703004:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626829056/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828360 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828335.703037:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828335.703043:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828335.703050:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626829056:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828335.703062:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626829184:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828335.703203:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828335.703223:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828335/real 1787828335]  req@ffff8cd0c3dab840 x1874671626829184/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828360 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828335.703257:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626829184/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828360 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828335.703267:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828335.703271:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828335.703275:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626829184:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828335.703282:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626829312:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828335.703310:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828335.703322:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828335/real 1787828335]  req@ffff8cd0c3daa580 x1874671626829312/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828360 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828335.703335:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626829312/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828360 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828335.703344:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828335.703347:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828335.703351:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626829312:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828335.894229:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828336.278186:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828336.918228:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828337.302343:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828337.942195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828338.326235:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828338.966206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828339.350295:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828339.990238:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828340.374285:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828340.822279:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828340.822296:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828340.822303:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828340.822318:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2226
00000100:00080000:1.0:1787828340.822333:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828340.822343:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828340.822438:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626829440/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828340.822475:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828340.822480:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828340.822483:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828340.822488:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2226
00000100:00080000:1.0:1787828340.822493:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828340.822497:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828340.822511:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828340.822528:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626829568/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828340.822561:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828340.822566:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787828340.822569:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828340.822573:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2226
00000100:00080000:1.0:1787828340.822577:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1787828340.822587:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626829440:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787828340.822605:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828340.822615:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828340.822631:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626829696/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828340.822662:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828340.822668:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828340.822670:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828340.822676:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2226
00000100:00080000:1.0:1787828340.822683:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828340.822688:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828340.822696:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787828340.822716:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787828340.822717:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626829824/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828340.822783:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828340/real 1787828340]  req@ffff8cd0d48f4000 x1874671626829440/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828365 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828340.822808:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626829440/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828365 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828340.822834:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828340.822840:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828340.822846:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626829440:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828340.822856:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626829568:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828340.822920:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828340.822939:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828340/real 1787828340]  req@ffff8cd0d48f7480 x1874671626829568/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828365 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828340.822955:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626829568/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828365 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828340.822965:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828340.822969:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828340.822973:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626829568:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828340.823001:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626829696:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828340.823047:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828340.823061:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828340/real 1787828340]  req@ffff8cd0d48f6580 x1874671626829696/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828365 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828340.823076:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626829696/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828365 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828340.823086:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828340.823090:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828340.823094:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626829696:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828340.823200:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626829824:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828340.823231:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828340.823247:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828340/real 1787828340]  req@ffff8cd0d48f4b40 x1874671626829824/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828365 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828340.823262:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626829824/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828365 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828340.823272:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828340.823275:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828340.823279:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626829824:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828341.014180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828341.398243:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828342.038145:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828342.422266:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828343.062241:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828343.446267:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828344.086204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828344.470272:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828345.110198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828345.494318:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828345.942264:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828345.942278:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828345.942286:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828345.942303:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2231
00000100:00080000:1.0:1787828345.942321:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828345.942334:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828345.942407:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626829952/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828345.942466:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828345.942469:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828345.942484:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828345.942489:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2231
00000100:00080000:1.0:1787828345.942493:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828345.942497:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828345.942505:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828345.942514:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626829952:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787828345.942519:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626830080/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828345.942538:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828345.942541:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787828345.942543:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828345.942546:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2231
00000100:00080000:1.0:1787828345.942549:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828345.942551:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828345.942554:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828345.942562:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626830208/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828345.942569:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828345.942573:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828345.942575:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828345.942577:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2231
00000100:00080000:1.0:1787828345.942579:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828345.942582:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828345.942585:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787828345.942586:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787828345.942600:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626830336/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828345.942624:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828345/real 1787828345]  req@ffff8cd0d48f7840 x1874671626829952/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828370 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828345.942644:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626829952/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828370 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828345.942656:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828345.942660:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828345.942665:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626829952:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828345.942689:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626830080:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828345.942740:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828345.942775:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828345/real 1787828345]  req@ffff8cd0d48f5a40 x1874671626830080/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828370 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828345.942794:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626830080/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828370 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828345.942806:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828345.942811:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828345.942815:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626830080:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828345.942823:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626830208:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828345.942850:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828345.942863:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828345/real 1787828345]  req@ffff8cd0d48f43c0 x1874671626830208/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828370 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828345.942878:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626830208/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828370 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828345.942888:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828345.942891:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828345.942895:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626830208:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828345.942902:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626830336:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828345.942923:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828345.942934:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828345/real 1787828345]  req@ffff8cd0d48f70c0 x1874671626830336/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828370 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828345.943004:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626830336/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828370 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828345.943015:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828345.943019:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828345.943023:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626830336:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828346.134221:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828346.518181:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828347.158174:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828347.542345:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828348.182182:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828348.566238:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828349.206198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828349.590358:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828350.230250:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828350.614172:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828351.062216:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828351.062226:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828351.062231:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828351.062242:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2236
00000100:00080000:1.0:1787828351.062254:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828351.062262:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828351.062302:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626830464/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828351.062331:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828351.062335:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828351.062337:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828351.062341:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2236
00000100:00080000:1.0:1787828351.062344:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828351.062347:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828351.062371:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828351.062383:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626830592/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828351.062396:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828351.062400:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787828351.062403:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828351.062406:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2236
00000100:00080000:1.0:1787828351.062411:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828351.062430:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828351.062437:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828351.062453:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626830720/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828351.062455:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626830464:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787828351.062478:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828351.062481:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828351.062483:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828351.062486:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2236
00000100:00080000:1.0:1787828351.062490:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828351.062494:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828351.062498:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828351.062508:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626830848/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828351.062551:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828351.062594:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828351/real 1787828351]  req@ffff8cd0d48f61c0 x1874671626830464/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828376 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828351.062613:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626830464/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828376 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828351.062624:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828351.062628:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828351.062633:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626830464:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828351.062641:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626830592:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828351.062661:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828351.062672:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828351/real 1787828351]  req@ffff8cd0d48f7c00 x1874671626830592/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828376 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828351.062684:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626830592/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828376 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828351.062701:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828351.062704:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828351.062708:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626830592:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828351.062734:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626830720:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828351.062756:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828351.062767:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828351/real 1787828351]  req@ffff8cd0d48f52c0 x1874671626830720/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828376 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828351.062778:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626830720/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828376 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828351.062785:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828351.062788:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828351.062790:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626830720:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828351.062796:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626830848:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828351.062811:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828351.062819:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828351/real 1787828351]  req@ffff8cd0d48f6d00 x1874671626830848/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828376 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828351.062830:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626830848/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828376 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828351.062837:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828351.062839:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828351.062903:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626830848:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828351.254174:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828351.638226:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828352.278181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828352.662299:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828353.302216:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828353.686216:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828354.326179:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828354.710216:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828355.350183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828355.734291:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828356.182219:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828356.182232:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828356.182238:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828356.182251:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2242
00000100:00080000:2.0:1787828356.182264:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828356.182274:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828356.182321:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626830976/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828356.182367:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828356.182372:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828356.182375:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828356.182379:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2242
00000100:00080000:2.0:1787828356.182383:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828356.182405:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828356.182421:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828356.182468:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab480 x1874671626831104/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828356.182476:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626830976:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828356.182497:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828356.182502:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828356.182505:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828356.182510:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2242
00000100:00080000:2.0:1787828356.182515:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828356.182520:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828356.182526:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828356.182572:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626831232/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828356.182586:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828356.182591:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828356.182597:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:3.0:1787828356.182599:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787828356.182606:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2242
00000100:00080000:2.0:1787828356.182614:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828356.182621:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828356.182629:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828356.182644:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626831360/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828356.182670:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828356/real 1787828356]  req@ffff8cd0c3da8780 x1874671626830976/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828381 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828356.182696:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626830976/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828381 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828356.182710:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828356.182715:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828356.182737:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626830976:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828356.182777:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626831104:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828356.182835:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828356.182861:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828356/real 1787828356]  req@ffff8cd0c3dab480 x1874671626831104/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828381 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828356.182886:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab480 x1874671626831104/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828381 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828356.182902:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828356.182908:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828356.182915:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626831104:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828356.182946:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626831232:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828356.182993:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828356.183043:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828356/real 1787828356]  req@ffff8cd0c3daa940 x1874671626831232/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828381 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828356.183069:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626831232/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828381 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828356.183127:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828356.183135:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828356.183142:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626831232:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828356.183171:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626831360:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828356.183217:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828356.183265:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828356/real 1787828356]  req@ffff8cd0c3dab0c0 x1874671626831360/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828381 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828356.183288:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626831360/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828381 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828356.183303:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828356.183310:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828356.183317:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626831360:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828356.374157:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828356.758314:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828357.398183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828357.782346:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828358.422192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828358.806196:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828359.446164:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828359.830288:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828360.470255:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828360.854144:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828361.302286:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828361.302295:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828361.302298:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828361.302307:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2247
00000100:00080000:1.0:1787828361.302348:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828361.302358:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828361.302411:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626831488/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828361.302465:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828361.302470:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828361.302492:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828361.302498:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2247
00000100:00080000:1.0:1787828361.302505:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828361.302526:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828361.302535:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828361.302548:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626831616/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828361.302549:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626831488:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787828361.302562:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828361.302570:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787828361.302572:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828361.302575:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2247
00000100:00080000:1.0:1787828361.302577:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828361.302579:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828361.302583:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828361.302590:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626831744/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828361.302597:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828361.302600:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828361.302602:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828361.302604:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2247
00000100:00080000:1.0:1787828361.302606:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828361.302609:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828361.302612:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787828361.302621:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787828361.302629:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626831872/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828361.302659:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828361/real 1787828361]  req@ffff8cd0d48f6d00 x1874671626831488/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828386 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828361.302673:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626831488/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828386 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828361.302681:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828361.302684:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828361.302687:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626831488:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828361.302727:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626831616:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828361.302743:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828361.302753:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828361/real 1787828361]  req@ffff8cd0d48f52c0 x1874671626831616/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828386 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828361.302763:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626831616/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828386 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828361.302772:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828361.302783:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828361.302787:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626831616:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828361.302793:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626831744:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828361.302826:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828361.302863:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828361/real 1787828361]  req@ffff8cd0d48f7c00 x1874671626831744/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828386 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828361.302930:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626831744/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828386 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828361.302946:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828361.302953:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828361.302960:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626831744:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828361.302973:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626831872:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828361.303046:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828361.303056:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828361/real 1787828361]  req@ffff8cd0d48f61c0 x1874671626831872/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828386 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828361.303064:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626831872/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828386 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828361.303092:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828361.303096:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828361.303139:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626831872:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828361.494193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828361.878325:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828362.518218:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828362.902225:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828363.542229:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828363.926253:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828364.566179:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828364.950163:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828365.590173:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828365.974308:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828366.422303:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828366.422315:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828366.422321:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828366.422333:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2252
00000100:00080000:2.0:1787828366.422347:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828366.422357:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828366.422404:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626832000/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828366.422441:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828366.422446:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828366.422449:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828366.422453:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2252
00000100:00080000:2.0:1787828366.422457:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828366.422468:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828366.422481:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828366.422495:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626832128/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828366.422559:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828366.422564:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828366.422568:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1787828366.422574:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626832000:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828366.422578:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2252
00000100:00080000:2.0:1787828366.422593:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828366.422601:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828366.422609:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828366.422626:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626832256/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828366.422639:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828366.422644:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828366.422646:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828366.422651:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2252
00000100:00080000:2.0:1787828366.422658:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828366.422662:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828366.422669:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787828366.422676:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828366.422691:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626832384/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828366.422741:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828366/real 1787828366]  req@ffff8cd0c3da9a40 x1874671626832000/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828391 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828366.422791:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626832000/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828391 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828366.422805:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828366.422811:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828366.422817:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626832000:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828366.422846:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626832128:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828366.422877:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828366.422894:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828366/real 1787828366]  req@ffff8cd0c3da8f00 x1874671626832128/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828391 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828366.422910:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626832128/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828391 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828366.422921:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828366.422925:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828366.422929:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626832128:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828366.422936:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626832256:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828366.422988:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828366.423002:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828366/real 1787828366]  req@ffff8cd0c3da9680 x1874671626832256/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828391 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828366.423017:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626832256/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828391 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828366.423044:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828366.423049:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828366.423053:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626832256:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828366.423062:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626832384:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828366.423131:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828366.423147:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828366/real 1787828366]  req@ffff8cd0c3da83c0 x1874671626832384/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828391 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828366.423162:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626832384/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828391 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828366.423172:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828366.423176:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828366.423180:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626832384:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828366.614199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828366.998244:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828367.638196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828368.022256:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828368.662175:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828369.046246:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828369.686207:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828370.070323:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828370.710246:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828371.094292:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828371.542161:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828371.542171:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828371.542175:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828371.542184:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2257
00000100:00080000:2.0:1787828371.542194:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828371.542201:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828371.542238:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8b40 x1874671626832512/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828371.542265:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828371.542268:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828371.542270:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828371.542272:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2257
00000100:00080000:2.0:1787828371.542275:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828371.542278:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828371.542286:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828371.542298:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626832640/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828371.542308:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828371.542311:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828371.542313:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828371.542316:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2257
00000100:00080000:2.0:1787828371.542319:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828371.542322:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828371.542325:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828371.542350:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626832768/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828371.542363:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626832512:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828371.542364:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828371.542374:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828371.542379:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828371.542385:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2257
00000100:00080000:2.0:1787828371.542389:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828371.542392:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828371.542396:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828371.542411:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626832896/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828371.542472:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828371.542534:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828371/real 1787828371]  req@ffff8cd0c3da8b40 x1874671626832512/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828396 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828371.542551:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8b40 x1874671626832512/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828396 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828371.542561:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828371.542576:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828371.542580:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626832512:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828371.542587:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626832640:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828371.542608:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828371.542619:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828371/real 1787828371]  req@ffff8cd0c3da83c0 x1874671626832640/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828396 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828371.542634:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626832640/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828396 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828371.542641:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828371.542643:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828371.542646:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626832640:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828371.542671:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626832768:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828371.542686:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828371.542699:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828371/real 1787828371]  req@ffff8cd0c3da9680 x1874671626832768/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828396 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828371.542708:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626832768/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828396 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828371.542714:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828371.542716:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828371.542718:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626832768:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828371.542722:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626832896:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828371.542743:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828371.542750:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828371/real 1787828371]  req@ffff8cd0c3da8f00 x1874671626832896/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828396 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828371.542758:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626832896/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828396 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828371.542764:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828371.542766:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828371.542768:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626832896:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828371.734168:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828372.118365:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828372.758262:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828373.142245:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828373.782145:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828374.166353:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828374.806211:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828375.190180:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828375.830259:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828376.214321:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828376.662278:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828376.662290:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828376.662296:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828376.662308:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2262
00000100:00080000:0.0:1787828376.662321:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828376.662331:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828376.662384:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626833024/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828376.662447:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828376.662454:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828376.662458:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828376.662465:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2262
00000100:00080000:0.0:1787828376.662472:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828376.662480:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828376.662494:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828376.662531:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626833024:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787828376.662548:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626833152/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828376.662566:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828376.662571:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828376.662574:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828376.662580:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2262
00000100:00080000:0.0:1787828376.662585:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828376.662589:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828376.662628:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787828376.662633:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787828376.662676:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626833280/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828376.662710:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828376/real 1787828376]  req@ffff8cd0c8e9f840 x1874671626833024/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828401 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787828376.662712:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828376.662727:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828376.662733:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828376.662760:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2262
00000100:00100000:3.0:1787828376.662767:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626833024/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828401 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787828376.662768:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828376.662782:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00080000:3.0:1787828376.662797:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:0.0:1787828376.662798:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00080000:3.0:1787828376.662810:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828376.662816:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626833024:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787828376.662825:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626833408/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828376.662852:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626833152:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828376.662913:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828376.662944:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828376/real 1787828376]  req@ffff8cd0c8e9f480 x1874671626833152/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828401 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828376.662961:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626833152/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828401 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828376.662972:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828376.662976:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828376.662980:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626833152:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828376.662987:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626833280:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828376.663011:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828376.663023:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828376/real 1787828376]  req@ffff8cd0c8e9cb40 x1874671626833280/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828401 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828376.663036:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626833280/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828401 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828376.663046:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828376.663049:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828376.663052:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626833280:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828376.663059:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626833408:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828376.663077:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828376.663144:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828376/real 1787828376]  req@ffff8cd0c8e9c000 x1874671626833408/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828401 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828376.663175:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626833408/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828401 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828376.663201:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828376.663209:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828376.663215:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626833408:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828376.854262:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828377.238206:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828377.878141:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828378.262281:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828378.902173:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828379.286180:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828379.926154:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828380.310297:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828380.950194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828381.334172:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828381.782277:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828381.782289:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828381.782295:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828381.782307:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2267
00000100:00080000:0.0:1787828381.782320:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828381.782330:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828381.782370:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d2c0 x1874671626833536/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828381.782400:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828381.782418:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828381.782420:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828381.782424:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2267
00000100:00080000:0.0:1787828381.782444:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828381.782447:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828381.782467:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828381.782478:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626833664/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828381.782487:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828381.782491:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828381.782492:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828381.782495:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2267
00000100:00080000:0.0:1787828381.782498:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828381.782501:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828381.782504:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828381.782514:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626833536:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787828381.782521:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626833792/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828381.782531:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828381.782533:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828381.782535:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828381.782538:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2267
00000100:00080000:0.0:1787828381.782540:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828381.782543:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828381.782553:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828381.782562:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626833920/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828381.782638:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626833664:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828381.782707:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626833792:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828381.782739:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626833920:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828381.974189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828382.358320:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828382.998207:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828383.382233:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828384.022232:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828384.406278:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:1.0:1787828384.854360:0:2343:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:1.0:1787828384.854398:0:2343:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1787828384.854412:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787828384.854449:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787828384.854459:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787828384.854470:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:3.0:1787828384.854503:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828381/real 1787828384]  req@ffff8cd0c8e9cb40 x1874671626833792/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828406 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828384.854530:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626833792/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828406 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828384.854542:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828384.854547:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828384.854552:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626833792:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828384.854560:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828381/real 1787828384]  req@ffff8cd0c8e9f480 x1874671626833920/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828406 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828384.854605:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626833920/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828406 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828384.854616:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828384.854620:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828384.854624:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626833920:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828384.854632:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828381/real 1787828384]  req@ffff8cd0c8e9d2c0 x1874671626833536/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828406 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828384.854646:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d2c0 x1874671626833536/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828406 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828384.854655:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828384.854659:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828384.854662:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626833536:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828384.854670:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828381/real 1787828384]  req@ffff8cd0c8e9c000 x1874671626833664/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828406 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828384.854683:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626833664/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828406 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828384.854694:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828384.854697:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828384.854701:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626833664:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828385.046190:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828385.430180:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828386.070154:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828386.454330:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828386.902277:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828386.902307:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828386.902313:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828386.902326:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2272
00000100:00080000:2.0:1787828386.902339:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828386.902349:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828386.902395:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626834048/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828386.902438:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828386.902446:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828386.902450:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828386.902459:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2272
00000100:00080000:2.0:1787828386.902466:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828386.902474:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828386.902492:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828386.902522:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626834176/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828386.902558:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626834048:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828386.902561:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828386.902581:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828386.902586:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828386.902596:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2272
00000100:00080000:2.0:1787828386.902604:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828386.902613:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828386.902625:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828386.902663:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626834304/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828386.902690:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787828386.902717:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828386.902725:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828386.902730:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828386.902740:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2272
00000100:00080000:2.0:1787828386.902751:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828386.902765:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828386.902765:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828386/real 1787828386]  req@ffff8cd0c3da9a40 x1874671626834048/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828411 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1787828386.902781:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828386.902843:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626834048/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828411 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828386.902848:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab480 x1874671626834432/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828386.902895:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828386.902909:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828386.902919:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626834048:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828386.902960:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626834176:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828386.903009:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828386.903037:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828386/real 1787828386]  req@ffff8cd0c3dab0c0 x1874671626834176/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828411 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828386.903065:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626834176/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828411 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828386.903082:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828386.903137:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828386.903146:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626834176:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828386.903159:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626834304:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828386.903205:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828386.903231:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828386/real 1787828386]  req@ffff8cd0c3daa940 x1874671626834304/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828411 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828386.903285:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626834304/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828411 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828386.903301:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828386.903308:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828386.903315:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626834304:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828386.903327:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626834432:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828386.903373:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828386.903398:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828386/real 1787828386]  req@ffff8cd0c3dab480 x1874671626834432/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828411 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828386.903422:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab480 x1874671626834432/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828411 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828386.903439:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828386.903445:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828386.903452:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626834432:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828387.094190:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828387.478245:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828388.118216:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828388.502329:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828389.142212:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828389.526262:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828390.166315:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828390.550277:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828391.190279:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828391.574248:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828392.022275:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828392.022289:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828392.022294:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828392.022307:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2277
00000100:00080000:0.0:1787828392.022322:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828392.022332:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828392.022383:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626834560/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828392.022431:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828392.022437:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828392.022439:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828392.022444:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2277
00000100:00080000:0.0:1787828392.022448:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828392.022459:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828392.022472:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828392.022489:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626834688/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828392.022509:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626834560:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828392.022537:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828392.022542:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828392.022546:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828392.022554:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2277
00000100:00080000:0.0:1787828392.022560:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828392.022565:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828392.022573:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828392.022604:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626834816/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828392.022610:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787828392.022626:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828392.022630:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828392.022633:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828392.022638:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2277
00000100:00080000:0.0:1787828392.022642:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828392.022648:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828392.022657:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828392.022675:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626834944/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828392.022958:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828392/real 1787828392]  req@ffff8cd0c8e9f840 x1874671626834560/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828417 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828392.023024:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626834560/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828417 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828392.023075:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828392.023083:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828392.023090:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626834560:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828392.023215:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626834688:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828392.023318:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828392.023349:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828392/real 1787828392]  req@ffff8cd0c8e9ed00 x1874671626834688/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828417 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828392.023374:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626834688/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828417 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828392.023387:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828392.023423:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828392.023428:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626834688:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828392.023438:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626834816:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828392.023475:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828392.023494:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828392/real 1787828392]  req@ffff8cd0c8e9c3c0 x1874671626834816/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828417 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828392.023519:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626834816/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828417 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828392.023551:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828392.023573:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828392.023580:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626834816:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828392.023621:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626834944:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828392.023658:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828392.023675:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828392/real 1787828392]  req@ffff8cd0c8e9d680 x1874671626834944/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828417 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828392.023691:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626834944/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828417 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828392.023702:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828392.023707:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828392.023711:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626834944:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828392.214221:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828392.598294:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828393.238260:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828393.622268:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828394.262193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828394.646268:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828395.286213:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828395.670282:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828396.310147:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828396.694322:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828397.142218:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828397.142240:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828397.142264:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828397.142278:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2282
00000100:00080000:0.0:1787828397.142294:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828397.142306:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828397.142359:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626835072/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828397.142400:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828397.142404:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828397.142407:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828397.142411:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2282
00000100:00080000:0.0:1787828397.142416:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828397.142433:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828397.142445:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828397.142463:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626835200/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828397.142476:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828397.142481:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828397.142485:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1787828397.142489:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626835072:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828397.142493:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2282
00000100:00080000:0.0:1787828397.142503:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828397.142515:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828397.142520:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828397.142535:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626835328/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828397.142563:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828397.142567:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828397.142570:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828397.142575:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2282
00000100:00080000:0.0:1787828397.142581:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1787828397.142587:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787828397.142594:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828397.142603:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828397.142618:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626835456/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828397.142645:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828397/real 1787828397]  req@ffff8cd0c8e9e940 x1874671626835072/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828422 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828397.142668:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626835072/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828422 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828397.142681:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828397.142685:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828397.142690:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626835072:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828397.142715:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626835200:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828397.142785:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828397.142806:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828397/real 1787828397]  req@ffff8cd0c8e9de00 x1874671626835200/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828422 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828397.142821:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626835200/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828422 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828397.142831:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828397.142835:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828397.142839:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626835200:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828397.142846:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626835328:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828397.142865:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828397.142886:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828397/real 1787828397]  req@ffff8cd0c8e9cf00 x1874671626835328/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828422 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828397.142899:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626835328/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828422 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828397.142907:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828397.142911:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828397.142914:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626835328:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828397.142920:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626835456:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828397.142936:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828397.142951:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828397/real 1787828397]  req@ffff8cd0c8e9fc00 x1874671626835456/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828422 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828397.142981:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626835456/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828422 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828397.142990:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828397.142994:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828397.142997:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626835456:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828397.334186:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828397.718260:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828398.358193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828398.742268:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828399.382165:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828399.766178:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828400.406160:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828400.790332:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828401.430199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828401.814240:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828402.262163:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828402.262171:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828402.262175:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828402.262183:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2288
00000100:00080000:0.0:1787828402.262192:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828402.262208:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828402.262251:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626835584/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828402.262283:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828402.262285:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828402.262286:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828402.262303:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2288
00000100:00080000:0.0:1787828402.262306:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828402.262309:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828402.262318:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828402.262330:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626835712/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828402.262336:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626835584:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828402.262341:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828402.262344:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828402.262346:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828402.262349:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2288
00000100:00080000:0.0:1787828402.262351:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828402.262353:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828402.262357:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828402.262364:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626835840/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828402.262371:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828402.262374:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828402.262375:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828402.262378:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2288
00000100:00080000:0.0:1787828402.262380:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828402.262382:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828402.262385:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828402.262393:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626835968/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828402.262403:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828402.262466:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828402/real 1787828402]  req@ffff8cd0c8e9e580 x1874671626835584/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828427 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828402.262480:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626835584/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828427 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828402.262488:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828402.262491:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828402.262494:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626835584:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828402.262518:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626835712:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828402.262543:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828402.262557:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828402/real 1787828402]  req@ffff8cd0c8e9fc00 x1874671626835712/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828427 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828402.262570:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626835712/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828427 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828402.262587:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828402.262591:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828402.262595:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626835712:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828402.262601:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626835840:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828402.262624:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828402.262635:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828402/real 1787828402]  req@ffff8cd0c8e9cf00 x1874671626835840/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828427 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828402.262646:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626835840/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828427 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828402.262654:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828402.262657:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828402.262661:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626835840:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828402.262667:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626835968:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828402.262684:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828402.262712:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828402/real 1787828402]  req@ffff8cd0c8e9de00 x1874671626835968/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828427 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828402.262726:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626835968/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828427 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828402.262734:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828402.262738:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828402.262747:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626835968:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828402.454178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828402.838223:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828403.478209:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828403.862201:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828404.502246:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828404.886287:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828405.526241:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828405.910248:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828406.550201:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828406.934327:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828407.382305:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828407.382319:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828407.382326:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828407.382345:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2293
00000100:00080000:0.0:1787828407.382365:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828407.382378:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828407.382448:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626836096/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828407.382492:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828407.382497:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828407.382499:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828407.382503:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2293
00000100:00080000:0.0:1787828407.382519:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828407.382524:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828407.382538:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828407.382555:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626836224/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828407.382568:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828407.382572:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828407.382574:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828407.382577:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2293
00000100:00100000:3.0:1787828407.382587:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626836096:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828407.382613:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828407.382620:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828407.382627:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828407.382642:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626836352/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828407.382652:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828407.382676:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828407.382679:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828407.382686:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2293
00000100:00080000:0.0:1787828407.382692:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828407.382714:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828407.382722:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828407.382737:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626836480/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828407.382751:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828407.382811:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828407/real 1787828407]  req@ffff8cd0c8e9e940 x1874671626836096/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828432 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828407.382831:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626836096/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828432 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828407.382847:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828407.382853:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828407.382859:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626836096:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828407.382870:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626836224:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828407.382904:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828407.382924:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828407/real 1787828407]  req@ffff8cd0c8e9d680 x1874671626836224/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828432 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828407.382941:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626836224/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828432 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828407.382953:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828407.382958:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828407.382961:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626836224:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828407.383015:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626836352:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828407.383039:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828407.383050:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828407/real 1787828407]  req@ffff8cd0c8e9c3c0 x1874671626836352/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828432 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828407.383060:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626836352/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828432 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828407.383077:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828407.383080:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828407.383108:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626836352:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828407.383114:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626836480:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828407.383135:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828407.383151:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828407/real 1787828407]  req@ffff8cd0c8e9ed00 x1874671626836480/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828432 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828407.383167:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626836480/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828432 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828407.383179:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828407.383183:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828407.383188:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626836480:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828407.574254:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828407.958262:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828408.598159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828408.982284:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828409.622276:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828410.006361:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828410.646219:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828411.030340:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828411.670214:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828412.054320:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828412.502235:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828412.502247:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828412.502253:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828412.502266:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2298
00000100:00080000:0.0:1787828412.502281:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828412.502290:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828412.502344:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626836608/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828412.502421:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828412.502427:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828412.502430:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828412.502437:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2298
00000100:00080000:0.0:1787828412.502444:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828412.502450:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828412.502464:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828412.502485:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626836736/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828412.502500:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828412.502504:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828412.502507:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828412.502514:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2298
00000100:00100000:3.0:1787828412.502515:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626836608:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828412.502519:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828412.502527:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828412.502533:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828412.502546:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d2c0 x1874671626836864/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828412.502558:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828412.502562:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828412.502563:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828412.502567:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2298
00000100:00080000:0.0:1787828412.502581:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828412.502586:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828412.502593:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828412.502607:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626836992/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828412.502673:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828412.502733:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828412/real 1787828412]  req@ffff8cd0c8e9f840 x1874671626836608/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828437 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828412.502755:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626836608/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828437 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828412.502769:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828412.502773:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828412.502778:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626836608:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828412.502808:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626836736:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828412.502834:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828412.502847:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828412/real 1787828412]  req@ffff8cd0c8e9c000 x1874671626836736/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828437 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828412.502860:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626836736/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828437 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828412.502868:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828412.502871:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828412.502874:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626836736:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828412.502880:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626836864:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828412.502912:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828412.502923:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828412/real 1787828412]  req@ffff8cd0c8e9d2c0 x1874671626836864/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828437 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828412.502952:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d2c0 x1874671626836864/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828437 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828412.502961:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828412.502964:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828412.502967:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626836864:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828412.502973:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626836992:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828412.502994:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828412.503004:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828412/real 1787828412]  req@ffff8cd0c8e9f480 x1874671626836992/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828437 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828412.503015:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626836992/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828437 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828412.503023:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828412.503026:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828412.503029:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626836992:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828412.694220:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828413.078293:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828413.718207:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828414.102276:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828414.742241:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828415.126315:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828415.766247:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828416.150202:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828416.790193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828417.174324:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828417.622202:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828417.622213:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828417.622218:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828417.622229:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2303
00000100:00080000:2.0:1787828417.622241:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828417.622248:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828417.622292:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626837120/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828417.622317:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828417.622320:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828417.622322:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828417.622324:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2303
00000100:00080000:2.0:1787828417.622327:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828417.622329:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828417.622339:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828417.622363:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626837248/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828417.622372:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626837120:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828417.622375:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828417.622383:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828417.622385:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828417.622390:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2303
00000100:00080000:2.0:1787828417.622394:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828417.622398:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828417.622404:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828417.622419:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626837376/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828417.622431:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828417.622435:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828417.622437:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828417.622442:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2303
00000100:00080000:2.0:1787828417.622448:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828417.622452:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828417.622456:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787828417.622467:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828417.622468:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626837504/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828417.622516:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828417/real 1787828417]  req@ffff8cd0c3da8780 x1874671626837120/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828442 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828417.622546:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626837120/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828442 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828417.622555:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828417.622559:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828417.622562:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626837120:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828417.622599:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626837248:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828417.622643:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828417.622658:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828417/real 1787828417]  req@ffff8cd0c3daa580 x1874671626837248/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828442 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828417.622666:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626837248/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828442 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828417.622672:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828417.622675:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828417.622677:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626837248:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828417.622681:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626837376:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828417.622704:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828417.622711:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828417/real 1787828417]  req@ffff8cd0c3dab840 x1874671626837376/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828442 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828417.622719:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626837376/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828442 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828417.622733:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828417.622736:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828417.622738:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626837376:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828417.622742:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626837504:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828417.622756:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828417.622773:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828417/real 1787828417]  req@ffff8cd0c3da8000 x1874671626837504/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828442 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828417.622780:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626837504/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828442 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828417.622785:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828417.622787:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828417.622789:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626837504:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828417.814189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828418.198294:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828418.838173:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828419.222366:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828419.862243:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828420.246345:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828420.886188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828421.270324:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828421.910173:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828422.294250:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828422.742203:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828422.742214:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828422.742218:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828422.742226:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2308
00000100:00080000:0.0:1787828422.742236:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828422.742243:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828422.742289:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626837632/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828422.742322:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828422.742327:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828422.742329:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828422.742334:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2308
00000100:00080000:0.0:1787828422.742338:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828422.742342:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828422.742351:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828422.742381:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626837760/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828422.742395:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626837632:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828422.742406:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828422.742412:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828422.742414:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828422.742422:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2308
00000100:00080000:0.0:1787828422.742429:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828422.742436:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828422.742443:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828422.742457:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d2c0 x1874671626837888/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828422.742477:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828422.742480:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828422.742482:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828422.742486:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2308
00000100:00080000:0.0:1787828422.742492:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828422.742496:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828422.742499:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787828422.742508:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787828422.742514:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626838016/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828422.742608:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828422/real 1787828422]  req@ffff8cd0c8e9cb40 x1874671626837632/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828447 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828422.742629:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626837632/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828447 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828422.742641:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828422.742669:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828422.742673:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626837632:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828422.742703:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626837760:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828422.742730:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828422.742743:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828422/real 1787828422]  req@ffff8cd0c8e9f480 x1874671626837760/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828447 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828422.742754:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626837760/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828447 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828422.742762:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828422.742765:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828422.742768:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626837760:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828422.742773:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626837888:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828422.742799:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828422.742811:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828422/real 1787828422]  req@ffff8cd0c8e9d2c0 x1874671626837888/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828447 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828422.742826:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d2c0 x1874671626837888/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828447 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828422.742837:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828422.742841:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828422.742846:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626837888:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828422.742856:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626838016:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828422.742914:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828422.742924:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828422/real 1787828422]  req@ffff8cd0c8e9c000 x1874671626838016/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828447 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828422.742933:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626838016/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828447 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828422.742939:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828422.742941:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828422.742943:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626838016:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828422.934201:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828423.318271:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787828423.958214:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828424.342178:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828424.982395:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828425.366373:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828426.006236:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828426.390208:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828427.030189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828427.414355:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828427.862271:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828427.862285:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828427.862292:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828427.862308:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2313
00000100:00080000:2.0:1787828427.862323:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828427.862334:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828427.862387:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626838144/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828427.862432:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828427.862437:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828427.862440:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828427.862444:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2313
00000100:00080000:2.0:1787828427.862451:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828427.862458:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828427.862472:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828427.862497:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626838272/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828427.862511:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626838144:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828427.862561:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828427.862571:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828427.862577:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828427.862590:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2313
00000100:00080000:2.0:1787828427.862601:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828427.862631:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828427.862643:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787828427.862666:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828427.862668:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626838400/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828427.862690:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828427.862696:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828427.862700:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828427.862711:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2313
00000100:00080000:2.0:1787828427.862719:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828427.862724:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828427.862733:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828427.862757:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626838528/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828427.862759:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828427/real 1787828427]  req@ffff8cd0c3dabc00 x1874671626838144/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828452 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828427.862866:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626838144/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828452 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828427.862887:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828427.862894:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828427.862900:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626838144:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828427.862983:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626838272:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828427.863022:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828427.863041:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828427/real 1787828427]  req@ffff8cd0c3da9e00 x1874671626838272/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828452 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828427.863058:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626838272/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828452 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828427.863069:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828427.863074:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828427.863078:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626838272:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828427.863147:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626838400:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828427.863193:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828427.863216:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828427/real 1787828427]  req@ffff8cd0c3da92c0 x1874671626838400/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828452 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828427.863233:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626838400/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828452 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828427.863244:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828427.863249:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828427.863253:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626838400:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828427.863261:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626838528:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828427.863287:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828427.863300:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828427/real 1787828427]  req@ffff8cd0c3daa1c0 x1874671626838528/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828452 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828427.863343:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626838528/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828452 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828427.863361:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828427.863368:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828427.863374:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626838528:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828428.054195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828428.438344:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828429.078248:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828429.462285:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828430.102230:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828430.486267:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828431.126194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828431.510307:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828432.150196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828432.534208:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828432.982269:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828432.982282:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828432.982287:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828432.982301:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2318
00000100:00080000:2.0:1787828432.982317:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828432.982329:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828432.982380:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daad00 x1874671626838656/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828432.982428:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828432.982432:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828432.982451:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828432.982456:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2318
00000100:00080000:2.0:1787828432.982461:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828432.982467:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828432.982498:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828432.982513:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626838656:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828432.982529:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626838784/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828432.982552:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828432.982558:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828432.982562:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828432.982570:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2318
00000100:00080000:2.0:1787828432.982578:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828432.982589:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:3.0:1787828432.982590:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:2.0:1787828432.982604:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828432.982623:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626838912/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828432.982636:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828432.982643:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:3.0:1787828432.982643:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828432/real 1787828432]  req@ffff8cd0c3daad00 x1874671626838656/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828457 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828432.982646:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828432.982660:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2318
00000100:00080000:2.0:1787828432.982672:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828432.982680:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828432.982683:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daad00 x1874671626838656/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828457 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1787828432.982687:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00080000:3.0:1787828432.982697:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828432.982706:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828432.982711:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626838656:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828432.982719:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626839040/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828432.982746:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626838784:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828432.982790:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828432.982804:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828432/real 1787828432]  req@ffff8cd0c3daa1c0 x1874671626838784/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828457 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828432.982818:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626838784/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828457 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828432.982827:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828432.982830:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828432.982834:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626838784:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828432.982840:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626838912:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828432.982856:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828432.982866:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828432/real 1787828432]  req@ffff8cd0c3da92c0 x1874671626838912/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828457 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828432.982877:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626838912/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828457 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828432.982885:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828432.982887:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828432.982890:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626838912:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828432.982896:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626839040:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828432.982910:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828432.982919:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828432/real 1787828432]  req@ffff8cd0c3da9e00 x1874671626839040/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828457 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828432.982946:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626839040/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828457 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828432.982954:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828432.982957:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828432.982961:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626839040:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828433.174202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828433.558318:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828434.198163:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828434.582217:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828435.222144:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828435.606388:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828436.246151:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828436.630258:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828437.270189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828437.654392:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828438.102228:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828438.102242:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828438.102248:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828438.102262:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2323
00000100:00080000:0.0:1787828438.102278:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828438.102287:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828438.102341:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626839168/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828438.102375:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828438.102379:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828438.102381:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828438.102385:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2323
00000100:00080000:0.0:1787828438.102389:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828438.102392:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828438.102406:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828438.102421:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626839296/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828438.102433:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828438.102437:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828438.102439:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828438.102463:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2323
00000100:00080000:0.0:1787828438.102470:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1787828438.102475:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626839168:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828438.102508:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828438.102519:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828438.102539:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626839424/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828438.102552:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828438.102556:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828438.102559:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828438.102563:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2323
00000100:00080000:0.0:1787828438.102570:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828438.102574:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828438.102580:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787828438.102587:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787828438.102597:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626839552/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828438.102654:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828438/real 1787828438]  req@ffff8cd0c8e9f840 x1874671626839168/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828463 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828438.102676:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626839168/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828463 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828438.102689:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828438.102693:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828438.102712:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626839168:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828438.102722:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626839296:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828438.102764:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828438.102779:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828438/real 1787828438]  req@ffff8cd0c8e9ed00 x1874671626839296/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828463 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828438.102792:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626839296/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828463 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828438.102801:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828438.102805:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828438.102808:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626839296:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828438.102843:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626839424:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828438.102863:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828438.102873:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828438/real 1787828438]  req@ffff8cd0c8e9c3c0 x1874671626839424/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828463 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828438.102885:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626839424/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828463 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828438.102906:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828438.102909:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828438.102912:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626839424:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828438.102919:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626839552:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828438.102937:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828438.102948:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828438/real 1787828438]  req@ffff8cd0c8e9d680 x1874671626839552/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828463 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828438.102989:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626839552/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828463 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828438.103003:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828438.103008:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828438.103014:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626839552:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828438.294168:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828438.678257:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828439.318292:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828439.702308:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828440.342239:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828440.726216:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828441.366282:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828441.750310:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828442.390251:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828442.774330:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828443.222258:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828443.222266:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828443.222269:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828443.222277:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2329
00000100:00080000:1.0:1787828443.222288:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828443.222294:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828443.222327:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626839680/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828443.222353:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828443.222356:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828443.222357:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828443.222360:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2329
00000100:00080000:1.0:1787828443.222363:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828443.222365:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828443.222375:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828443.222389:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626839808/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828443.222404:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626839680:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787828443.222438:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828443.222443:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787828443.222445:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828443.222450:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2329
00000100:00080000:1.0:1787828443.222454:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828443.222458:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828443.222463:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828443.222474:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626839936/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828443.222482:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787828443.222485:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828443.222488:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828443.222490:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828443.222494:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2329
00000100:00080000:1.0:1787828443.222498:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828443.222501:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828443.222505:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828443.222513:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626840064/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828443.222532:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828443/real 1787828443]  req@ffff8cd0d48f70c0 x1874671626839680/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828468 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828443.222553:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626839680/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828468 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828443.222562:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828443.222565:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828443.222568:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626839680:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828443.222588:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626839808:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828443.222616:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828443.222625:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828443/real 1787828443]  req@ffff8cd0d48f43c0 x1874671626839808/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828468 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828443.222634:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626839808/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828468 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828443.222640:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828443.222643:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828443.222645:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626839808:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828443.222649:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626839936:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828443.222660:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828443.222666:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828443/real 1787828443]  req@ffff8cd0d48f5a40 x1874671626839936/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828468 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828443.222674:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626839936/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828468 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828443.222679:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828443.222681:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828443.222683:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626839936:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828443.222686:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626840064:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828443.222695:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828443.222701:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828443/real 1787828443]  req@ffff8cd0d48f7840 x1874671626840064/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828468 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828443.222719:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626840064/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828468 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828443.222725:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828443.222727:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828443.222729:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626840064:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828443.414155:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828443.798306:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828444.438210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828444.822261:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828445.462150:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828445.846270:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828446.486230:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828446.870195:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828447.510181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828447.894285:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828448.342339:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828448.342354:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828448.342361:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828448.342400:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2334
00000100:00080000:2.0:1787828448.342435:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828448.342444:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828448.342499:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626840192/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828448.342533:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828448.342537:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828448.342539:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828448.342543:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2334
00000100:00080000:2.0:1787828448.342547:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828448.342551:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828448.342571:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828448.342586:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626840320/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828448.342602:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787828448.342603:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626840192:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828448.342611:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828448.342618:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828448.342623:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2334
00000100:00080000:2.0:1787828448.342627:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828448.342654:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828448.342661:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828448.342678:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626840448/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828448.342690:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828448.342694:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828448.342696:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828448.342701:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2334
00000100:00080000:2.0:1787828448.342706:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828448.342710:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828448.342715:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828448.342728:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626840576/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828448.342772:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626840320:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828448.342815:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626840448:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828448.342850:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626840576:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828448.534256:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828448.918229:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828449.558184:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828449.942306:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828450.582241:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828450.966282:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:0.0:1787828451.414667:0:2345:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:0.0:1787828451.414721:0:2345:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:0.0:1787828451.414739:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:0.0:1787828451.414777:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:0.0:1787828451.414784:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:0.0:1787828451.414826:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:3.0:1787828451.414843:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828448/real 1787828451]  req@ffff8cd0c3da8000 x1874671626840320/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828473 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828451.414892:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626840320/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828473 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828451.414907:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828451.414913:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828451.414917:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626840320:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828451.414926:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828448/real 1787828451]  req@ffff8cd0c3dab840 x1874671626840448/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828473 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828451.414954:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626840448/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828473 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828451.414964:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828451.414969:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828451.414973:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626840448:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828451.414982:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828448/real 1787828451]  req@ffff8cd0c3daa580 x1874671626840576/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828473 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828451.414999:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626840576/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828473 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828451.415009:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828451.415014:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828451.415018:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626840576:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828451.415026:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828448/real 1787828451]  req@ffff8cd0c3dabc00 x1874671626840192/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828473 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828451.415053:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626840192/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828473 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828451.415063:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828451.415067:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828451.415077:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626840192:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828451.606237:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828451.990322:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828452.630206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828453.014302:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828453.462362:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828453.462377:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828453.462383:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828453.462397:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2339
00000100:00080000:1.0:1787828453.462434:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828453.462444:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828453.462498:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626840704/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828453.462546:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828453.462552:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828453.462555:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828453.462561:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2339
00000100:00080000:1.0:1787828453.462568:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828453.462576:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828453.462595:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828453.462628:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626840704:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787828453.462635:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626840832/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828453.462662:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828453.462667:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787828453.462672:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828453.462689:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2339
00000100:00080000:1.0:1787828453.462698:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828453.462706:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828453.462717:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828453.462741:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626840960/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828453.462760:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:3.0:1787828453.462762:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787828453.462780:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828453.462784:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828453.462794:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2339
00000100:00080000:1.0:1787828453.462803:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828453.462827:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828453.462836:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828453.462851:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828453/real 1787828453]  req@ffff8cd0d48f4b40 x1874671626840704/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828478 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787828453.462859:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626841088/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828453.462896:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626840704/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828478 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828453.462925:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828453.462932:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828453.462938:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626840704:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828453.462977:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626840832:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828453.463014:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828453.463032:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828453/real 1787828453]  req@ffff8cd0d48f6580 x1874671626840832/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828478 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828453.463049:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626840832/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828478 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828453.463071:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828453.463076:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828453.463081:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626840832:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828453.463742:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626840960:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828453.463791:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828453.463813:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828453/real 1787828453]  req@ffff8cd0d48f7480 x1874671626840960/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828478 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828453.463832:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626840960/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828478 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828453.463844:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828453.463849:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828453.463853:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626840960:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828453.463862:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626841088:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828453.463892:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828453.463909:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828453/real 1787828453]  req@ffff8cd0d48f4000 x1874671626841088/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828478 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828453.463925:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626841088/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828478 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828453.463936:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828453.463940:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828453.463958:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626841088:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828453.654235:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828454.038407:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828454.678240:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828455.062254:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828455.702214:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828456.086268:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828456.726214:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828457.110281:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828457.750159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828458.134388:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828458.582424:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828458.582438:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828458.582445:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828458.582460:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2344
00000100:00080000:3.0:1787828458.582478:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828458.582488:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828458.582561:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626841216/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828458.582621:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828458.582629:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828458.582633:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828458.582641:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2344
00000100:00080000:3.0:1787828458.582659:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828458.582670:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828458.582691:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828458.582720:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626841344/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828458.582729:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626841216:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787828458.582790:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828458.582795:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828458.582798:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828458.582806:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2344
00000100:00080000:3.0:1787828458.582816:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828458.582823:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828458.582832:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787828458.582842:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828458.582858:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626841472/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828458.582874:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828458.582879:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828458.582882:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828458.582888:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2344
00000100:00080000:3.0:1787828458.582896:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828458.582902:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828458.582910:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828458.582920:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828458/real 1787828458]  req@ffff8cd0c8eae1c0 x1874671626841216/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828483 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828458.582926:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626841600/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828458.582996:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626841216/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828483 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828458.583019:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828458.583027:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828458.583033:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626841216:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828458.583154:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626841344:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828458.583196:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828458.583231:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828458/real 1787828458]  req@ffff8cd0c8eac780 x1874671626841344/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828483 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828458.583247:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626841344/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828483 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828458.583257:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828458.583262:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828458.583266:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626841344:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828458.583274:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626841472:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828458.583296:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828458.583352:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828458/real 1787828458]  req@ffff8cd0c8eada40 x1874671626841472/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828483 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828458.583372:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626841472/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828483 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828458.583385:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828458.583390:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828458.583395:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626841472:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828458.583423:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626841600:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828458.583450:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828458.583465:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828458/real 1787828458]  req@ffff8cd0c8eaf840 x1874671626841600/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828483 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828458.583492:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626841600/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828483 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828458.583503:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828458.583508:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828458.583513:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626841600:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828458.774206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828459.158292:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828459.798203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828460.182260:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828460.822216:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828461.206173:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828461.846233:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828462.230340:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828462.870146:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828463.254221:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828463.702221:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828463.702235:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828463.702242:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828463.702255:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2349
00000100:00080000:3.0:1787828463.702311:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828463.702319:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828463.702364:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626841728/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828463.702420:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828463.702425:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828463.702428:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828463.702433:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2349
00000100:00080000:3.0:1787828463.702453:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828463.702458:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828463.702471:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828463.702489:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626841856/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828463.702498:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626841728:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787828463.702506:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828463.702511:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828463.702513:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828463.702518:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2349
00000100:00080000:3.0:1787828463.702522:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828463.702526:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828463.702551:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828463.702562:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626841984/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828463.702569:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828463.702572:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828463.702573:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828463.702576:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2349
00000100:00080000:3.0:1787828463.702580:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1787828463.702581:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787828463.702585:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828463.702590:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828463.702598:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626842112/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828463.702639:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828463/real 1787828463]  req@ffff8cd0c8eac3c0 x1874671626841728/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828488 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828463.702656:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626841728/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828488 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828463.702672:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828463.702676:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828463.702679:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626841728:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828463.702702:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626841856:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828463.702720:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828463.702729:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828463/real 1787828463]  req@ffff8cd0c8eaf0c0 x1874671626841856/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828488 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828463.702737:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626841856/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828488 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828463.702754:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828463.702757:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828463.702759:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626841856:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828463.702764:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626841984:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828463.702780:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828463.702787:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828463/real 1787828463]  req@ffff8cd0c8eaf840 x1874671626841984/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828488 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828463.702795:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626841984/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828488 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828463.702800:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828463.702802:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828463.702804:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626841984:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828463.702807:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626842112:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828463.702836:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828463.702848:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828463/real 1787828463]  req@ffff8cd0c8eada40 x1874671626842112/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828488 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828463.702861:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626842112/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828488 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828463.702881:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828463.702883:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828463.702891:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626842112:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828463.894155:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828464.278233:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828464.918202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828465.302203:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828465.942247:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828466.326305:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828466.966138:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828467.350215:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828467.990176:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828468.374256:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828468.822206:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828468.822214:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828468.822218:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828468.822227:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2354
00000100:00080000:2.0:1787828468.822253:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828468.822259:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828468.822293:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626842240/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828468.822320:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828468.822323:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828468.822325:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828468.822327:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2354
00000100:00080000:2.0:1787828468.822330:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828468.822334:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828468.822343:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828468.822356:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626842368/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828468.822357:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626842240:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828468.822373:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828468.822376:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828468.822378:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828468.822381:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2354
00000100:00080000:2.0:1787828468.822383:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828468.822402:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828468.822408:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828468.822418:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626842496/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828468.822424:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787828468.822433:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828468.822436:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828468.822438:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828468.822442:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2354
00000100:00080000:2.0:1787828468.822447:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828468.822450:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828468.822454:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828468.822464:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626842624/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828468.822471:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828468/real 1787828468]  req@ffff8cd0c3daa940 x1874671626842240/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828493 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828468.822488:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626842240/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828493 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828468.822508:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828468.822514:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828468.822519:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626842240:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828468.822546:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626842368:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828468.822575:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828468.822590:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828468/real 1787828468]  req@ffff8cd0c3dab0c0 x1874671626842368/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828493 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828468.822606:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626842368/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828493 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828468.822617:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828468.822621:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828468.822625:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626842368:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828468.822632:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626842496:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828468.822651:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828468.822662:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828468/real 1787828468]  req@ffff8cd0c3da9a40 x1874671626842496/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828493 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828468.822672:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626842496/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828493 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828468.822678:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828468.822681:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828468.822683:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626842496:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828468.822694:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626842624:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828468.822711:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828468.822719:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828468/real 1787828468]  req@ffff8cd0c3da8f00 x1874671626842624/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828493 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828468.822728:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626842624/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828493 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828468.822734:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828468.822736:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828468.822739:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626842624:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828469.014173:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828469.398274:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828470.038289:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828470.422351:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828471.062238:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828471.446190:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828472.086282:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828472.470271:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828473.110280:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828473.494264:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828473.942314:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828473.942325:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828473.942329:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828473.942341:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2359
00000100:00080000:3.0:1787828473.942354:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828473.942361:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828473.942420:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626842752/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828473.942489:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828473.942493:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828473.942495:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828473.942499:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2359
00000100:00080000:3.0:1787828473.942504:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828473.942508:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828473.942536:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828473.942561:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626842880/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828473.942570:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626842752:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787828473.942601:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828473.942609:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828473.942615:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828473.942624:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2359
00000100:00080000:3.0:1787828473.942633:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828473.942641:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828473.942654:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828473.942695:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626843008/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787828473.942700:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787828473.942720:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828473.942725:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828473.942727:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828473.942734:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2359
00000100:00080000:3.0:1787828473.942745:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828473.942750:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828473.942757:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828473.942765:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828473/real 1787828473]  req@ffff8cd0c8eac780 x1874671626842752/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828498 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828473.942771:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626843136/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828473.942853:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626842752/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828498 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828473.942869:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828473.942875:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828473.942880:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626842752:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828473.942941:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626842880:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828473.943008:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828473.943035:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828473/real 1787828473]  req@ffff8cd0c8eae1c0 x1874671626842880/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828498 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828473.943048:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626842880/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828498 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828473.943059:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828473.943064:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828473.943069:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626842880:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828473.943078:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626843008:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828473.943143:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828473.943156:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828473/real 1787828473]  req@ffff8cd0c8ead2c0 x1874671626843008/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828498 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828473.943167:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626843008/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828498 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828473.943175:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828473.943178:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828473.943181:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626843008:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828473.943187:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626843136:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828473.943233:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828473.943244:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828473/real 1787828473]  req@ffff8cd0c8eae940 x1874671626843136/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828498 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828473.943275:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626843136/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828498 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828473.943283:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828473.943286:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828473.943288:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626843136:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828474.134210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828474.518341:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828475.158205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828475.542223:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828476.182199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828476.566305:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828477.206198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828477.590239:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828478.230179:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828478.614319:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828479.062255:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828479.062263:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828479.062267:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828479.062275:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2364
00000100:00080000:2.0:1787828479.062286:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828479.062293:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828479.062328:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626843264/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828479.062356:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828479.062359:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828479.062361:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828479.062363:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2364
00000100:00080000:2.0:1787828479.062366:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828479.062369:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828479.062377:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828479.062385:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626843392/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828479.062425:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828479.062429:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828479.062432:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1787828479.062435:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626843264:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828479.062438:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2364
00000100:00080000:2.0:1787828479.062446:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828479.062449:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828479.062455:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828479.062467:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8b40 x1874671626843520/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828479.062474:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828479.062477:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828479.062479:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828479.062482:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2364
00000100:00080000:2.0:1787828479.062493:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828479.062496:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828479.062500:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787828479.062505:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828479.062512:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626843648/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828479.062549:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828479/real 1787828479]  req@ffff8cd0c3da9680 x1874671626843264/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828504 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828479.062565:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626843264/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828504 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828479.062573:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828479.062576:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828479.062579:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626843264:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828479.062599:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626843392:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828479.062615:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828479.062635:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828479/real 1787828479]  req@ffff8cd0c3da83c0 x1874671626843392/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828504 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828479.062653:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626843392/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828504 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828479.062659:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828479.062662:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828479.062664:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626843392:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828479.062669:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626843520:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828479.062683:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828479.062689:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828479/real 1787828479]  req@ffff8cd0c3da8b40 x1874671626843520/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828504 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828479.062698:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8b40 x1874671626843520/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828504 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828479.062718:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828479.062721:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828479.062725:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626843520:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828479.062732:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626843648:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828479.062755:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828479.062768:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828479/real 1787828479]  req@ffff8cd0c3da8f00 x1874671626843648/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828504 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828479.062789:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626843648/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828504 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828479.062804:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828479.062807:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828479.062809:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626843648:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828479.254283:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828479.638208:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828480.278219:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828480.662347:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828481.302192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828481.686252:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828482.326203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828482.710327:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828483.350217:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828483.734227:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828484.182221:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828484.182234:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828484.182239:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828484.182250:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2370
00000100:00080000:2.0:1787828484.182284:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828484.182293:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828484.182337:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626843776/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828484.182384:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828484.182388:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828484.182391:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828484.182396:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2370
00000100:00080000:2.0:1787828484.182400:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828484.182404:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828484.182417:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828484.182432:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626843904/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828484.182447:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787828484.182451:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626843776:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828484.182453:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828484.182459:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828484.182472:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2370
00000100:00080000:2.0:1787828484.182476:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828484.182480:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828484.182486:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828484.182500:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626844032/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828484.182512:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828484.182537:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828484.182541:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828484.182547:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2370
00000100:00080000:2.0:1787828484.182554:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1787828484.182560:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787828484.182566:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828484.182577:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828484.182595:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626844160/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828484.182622:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828484/real 1787828484]  req@ffff8cd0c3da9a40 x1874671626843776/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828509 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828484.182656:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626843776/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828509 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828484.182670:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828484.182675:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828484.182680:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626843776:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828484.182707:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626843904:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828484.182736:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828484.182752:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828484/real 1787828484]  req@ffff8cd0c3dab0c0 x1874671626843904/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828509 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828484.182788:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626843904/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828509 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828484.182798:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828484.182815:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828484.182820:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626843904:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828484.182828:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626844032:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828484.182856:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828484.182870:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828484/real 1787828484]  req@ffff8cd0c3daa940 x1874671626844032/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828509 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828484.182884:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626844032/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828509 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828484.182894:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828484.182897:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828484.182913:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626844032:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828484.182924:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626844160:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828484.182958:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828484.182980:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828484/real 1787828484]  req@ffff8cd0c3da8780 x1874671626844160/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828509 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828484.183013:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626844160/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828509 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828484.183045:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828484.183049:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828484.183054:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626844160:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828484.374203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828484.758410:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828485.398190:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828485.782260:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828486.422207:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828486.806357:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828487.446164:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828487.830235:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828488.470214:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828488.854343:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828489.302267:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828489.302279:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828489.302284:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828489.302297:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2375
00000100:00080000:2.0:1787828489.302311:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828489.302319:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828489.302372:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626844288/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828489.302400:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828489.302404:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828489.302406:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828489.302410:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2375
00000100:00080000:2.0:1787828489.302413:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828489.302417:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828489.302427:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828489.302438:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626844416/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828489.302468:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828489.302472:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828489.302474:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1787828489.302482:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626844288:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828489.302483:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2375
00000100:00080000:2.0:1787828489.302488:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828489.302498:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828489.302504:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828489.302518:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626844544/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828489.302537:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828489.302541:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828489.302543:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828489.302547:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2375
00000100:00080000:2.0:1787828489.302563:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828489.302568:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828489.302572:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828489.302587:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626844672/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828489.302591:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828489.302654:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828489/real 1787828489]  req@ffff8cd0c3dabc00 x1874671626844288/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828514 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828489.302681:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626844288/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828514 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828489.302697:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828489.302703:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828489.302709:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626844288:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828489.302739:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626844416:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828489.302776:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828489.302796:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828489/real 1787828489]  req@ffff8cd0c3daa580 x1874671626844416/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828514 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828489.302837:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626844416/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828514 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828489.302850:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828489.302855:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828489.302860:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626844416:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828489.302869:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626844544:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828489.302899:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828489.302914:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828489/real 1787828489]  req@ffff8cd0c3dab840 x1874671626844544/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828514 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828489.302933:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626844544/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828514 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828489.302944:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828489.302949:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828489.302955:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626844544:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828489.302965:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626844672:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828489.303005:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828489.303022:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828489/real 1787828489]  req@ffff8cd0c3da8000 x1874671626844672/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828514 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828489.303040:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626844672/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828514 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828489.303071:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828489.303076:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828489.303082:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626844672:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828489.494256:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828489.878215:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828490.518152:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828490.902200:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828491.542233:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828491.926217:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828492.566180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828492.950306:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828493.590237:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828493.974230:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828494.422215:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828494.422231:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828494.422238:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828494.422250:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2380
00000100:00080000:2.0:1787828494.422288:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828494.422297:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828494.422342:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626844800/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828494.422383:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828494.422388:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828494.422390:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828494.422413:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2380
00000100:00080000:2.0:1787828494.422417:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828494.422422:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828494.422435:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828494.422450:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626844928/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828494.422466:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828494.422471:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828494.422474:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828494.422478:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2380
00000100:00080000:2.0:1787828494.422483:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828494.422488:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828494.422496:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828494.422504:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626844800:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828494.422516:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626845056/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828494.422534:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828494.422548:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828494.422552:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828494.422558:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2380
00000100:00080000:2.0:1787828494.422580:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828494.422587:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828494.422595:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787828494.422601:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828494.422623:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daad00 x1874671626845184/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828494.422691:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828494/real 1787828494]  req@ffff8cd0c3da9e00 x1874671626844800/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828519 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828494.422726:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626844800/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828519 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828494.422740:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828494.422762:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828494.422768:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626844800:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828494.422779:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626844928:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828494.422816:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828494.422834:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828494/real 1787828494]  req@ffff8cd0c3da92c0 x1874671626844928/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828519 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828494.422873:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626844928/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828519 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828494.422902:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828494.422906:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828494.422911:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626844928:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828494.422943:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626845056:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828494.422973:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828494.422989:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828494/real 1787828494]  req@ffff8cd0c3daa1c0 x1874671626845056/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828519 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828494.423004:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626845056/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828519 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828494.423014:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828494.423017:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828494.423021:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626845056:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828494.423028:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626845184:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828494.423048:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828494.423159:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828494/real 1787828494]  req@ffff8cd0c3daad00 x1874671626845184/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828519 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828494.423185:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daad00 x1874671626845184/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828519 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828494.423220:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828494.423225:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828494.423230:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626845184:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828494.614205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828494.998338:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828495.638165:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828496.022290:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828496.662210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828497.046311:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828497.686189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828498.070218:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828498.710262:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828499.094281:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828499.542223:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828499.542237:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828499.542244:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828499.542260:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2385
00000100:00080000:2.0:1787828499.542277:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828499.542287:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828499.542350:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daad00 x1874671626845312/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828499.542392:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828499.542397:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828499.542400:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828499.542404:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2385
00000100:00080000:2.0:1787828499.542409:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828499.542414:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828499.542428:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828499.542445:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626845440/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828499.542498:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787828499.542503:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626845312:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828499.542506:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828499.542516:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828499.542530:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2385
00000100:00080000:2.0:1787828499.542539:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828499.542547:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828499.542561:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828499.542586:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626845568/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828499.542601:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828499.542614:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000800:00000100:3.0:1787828499.542614:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787828499.542623:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828499.542631:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2385
00000100:00080000:2.0:1787828499.542638:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828499.542645:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828499.542652:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828499.542667:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626845696/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828499.542690:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828499/real 1787828499]  req@ffff8cd0c3daad00 x1874671626845312/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828524 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828499.542716:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daad00 x1874671626845312/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828524 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828499.542730:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828499.542736:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828499.542740:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626845312:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828499.542770:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626845440:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828499.542799:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828499.542815:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828499/real 1787828499]  req@ffff8cd0c3daa1c0 x1874671626845440/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828524 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828499.542844:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626845440/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828524 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828499.542854:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828499.542858:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828499.542862:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626845440:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828499.542869:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626845568:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828499.542894:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828499.542907:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828499/real 1787828499]  req@ffff8cd0c3da92c0 x1874671626845568/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828524 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828499.542919:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626845568/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828524 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828499.542929:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828499.542932:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828499.542935:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626845568:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828499.542943:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626845696:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828499.542965:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828499.542978:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828499/real 1787828499]  req@ffff8cd0c3da9e00 x1874671626845696/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828524 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828499.542990:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626845696/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828524 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828499.543011:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828499.543032:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828499.543037:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626845696:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828499.734174:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828500.118172:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828500.758168:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828501.142324:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828501.782152:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828502.166228:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828502.806156:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828503.190392:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828503.830157:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828504.214176:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828504.662341:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828504.662356:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828504.662362:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828504.662378:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2390
00000100:00080000:3.0:1787828504.662409:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828504.662424:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828504.662500:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626845824/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828504.662561:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828504.662566:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828504.662569:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828504.662575:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2390
00000100:00080000:3.0:1787828504.662581:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828504.662586:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828504.662629:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828504.662650:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626845952/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828504.662668:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828504.662676:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:2.0:1787828504.662680:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626845824:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787828504.662681:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828504.662695:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2390
00000100:00080000:3.0:1787828504.662701:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828504.662706:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828504.662713:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828504.662732:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626846080/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828504.662746:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828504.662752:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828504.662754:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828504.662759:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2390
00000100:00080000:3.0:1787828504.662777:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828504.662783:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828504.662789:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787828504.662806:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828504.662807:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626846208/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828504.662894:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828504/real 1787828504]  req@ffff8cd0c8eacf00 x1874671626845824/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828529 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828504.662945:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626845824/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828529 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828504.662962:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828504.662967:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828504.662973:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626845824:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828504.663048:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626845952:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828504.663116:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828504.663137:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828504/real 1787828504]  req@ffff8cd0c8eac000 x1874671626845952/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828529 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828504.663154:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626845952/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828529 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828504.663165:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828504.663169:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828504.663174:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626845952:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828504.663182:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626846080:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828504.663207:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828504.663220:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828504/real 1787828504]  req@ffff8cd0c8eae940 x1874671626846080/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828529 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828504.663234:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626846080/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828529 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828504.663268:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828504.663273:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828504.663277:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626846080:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828504.663285:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626846208:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828504.663312:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828504.663332:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828504/real 1787828504]  req@ffff8cd0c8ead2c0 x1874671626846208/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828529 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828504.663346:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626846208/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828529 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828504.663356:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828504.663359:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828504.663363:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626846208:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828504.854144:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828505.238420:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828505.878205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828506.262204:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828506.902152:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828507.286292:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828507.926234:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828508.310184:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828508.950202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828509.334254:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828509.782195:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828509.782205:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828509.782209:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828509.782219:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2395
00000100:00080000:2.0:1787828509.782230:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828509.782238:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828509.782279:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626846336/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828509.782339:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828509.782370:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828509.782376:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828509.782385:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2395
00000100:00080000:2.0:1787828509.782394:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1787828509.782406:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626846336:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828509.782417:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828509.782439:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828509.782470:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626846464/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828509.782489:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828509.782519:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828509.782521:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:3.0:1787828509.782527:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787828509.782529:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2395
00000100:00080000:2.0:1787828509.782536:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828509.782541:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828509.782548:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828509.782559:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626846592/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828509.782568:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828509.782572:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828509.782574:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828509.782578:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2395
00000100:00080000:2.0:1787828509.782599:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828509.782603:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828509.782606:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828509/real 1787828509]  req@ffff8cd0c3da8000 x1874671626846336/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828534 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1787828509.782617:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828509.782628:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626846336/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828534 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828509.782629:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626846720/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828509.782658:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828509.782663:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828509.782667:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626846336:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828509.782686:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626846464:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828509.782703:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828509.782711:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828509/real 1787828509]  req@ffff8cd0c3dab840 x1874671626846464/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828534 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828509.782719:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626846464/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828534 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828509.782725:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828509.782727:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828509.782729:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626846464:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828509.782740:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626846592:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828509.782756:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828509.782763:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828509/real 1787828509]  req@ffff8cd0c3daa580 x1874671626846592/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828534 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828509.782771:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626846592/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828534 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828509.782776:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828509.782778:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828509.782781:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626846592:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828509.782784:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626846720:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828509.782794:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828509.782800:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828509/real 1787828509]  req@ffff8cd0c3dabc00 x1874671626846720/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828534 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828509.782816:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626846720/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828534 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828509.782821:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828509.782823:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828509.782824:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626846720:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828509.974276:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828510.358215:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828510.998248:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828511.382291:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828512.022225:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828512.406267:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828513.046273:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828513.430258:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828514.070218:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828514.454226:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828514.902259:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828514.902274:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828514.902282:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828514.902297:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2400
00000100:00080000:2.0:1787828514.902316:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828514.902327:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828514.902390:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626846848/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828514.902434:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828514.902439:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828514.902442:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828514.902446:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2400
00000100:00080000:2.0:1787828514.902451:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828514.902455:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828514.902470:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828514.902487:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626846976/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828514.902503:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828514.902508:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828514.902511:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828514.902515:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2400
00000100:00100000:3.0:1787828514.902542:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626846848:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828514.902555:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828514.902566:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828514.902575:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828514.902596:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626847104/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828514.902613:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828514.902618:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828514.902621:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828514.902627:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2400
00000100:00080000:2.0:1787828514.902646:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828514.902651:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828514.902657:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828514.902672:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626847232/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828514.902732:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626846976:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828514.902799:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626847104:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828514.902830:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626847232:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828515.094197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828515.478288:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828516.118227:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828516.502254:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828517.142183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828517.526315:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:1.0:1787828517.974595:0:2344:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:1.0:1787828517.974685:0:2344:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1787828517.974710:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787828517.974768:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787828517.974779:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787828517.974788:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:3.0:1787828517.974949:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828514/real 1787828517]  req@ffff8cd0c3dab0c0 x1874671626847104/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828517.974996:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626847104/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828517.975017:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828517.975026:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828517.975033:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626847104:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828517.975046:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828514/real 1787828517]  req@ffff8cd0c3da9a40 x1874671626847232/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828517.975069:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626847232/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828517.975341:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828517.975351:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828517.975358:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626847232:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828517.975373:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828514/real 1787828517]  req@ffff8cd0c3da8780 x1874671626846848/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828517.975397:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626846848/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828517.975410:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828517.975415:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828517.975421:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626846848:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828517.975432:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828514/real 1787828517]  req@ffff8cd0c3daa940 x1874671626846976/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828517.975451:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626846976/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828517.975463:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828517.975469:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828517.975474:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626846976:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828518.166285:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828518.550262:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828519.190191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828519.574246:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828520.022203:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828520.022216:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828520.022221:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828520.022235:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2405
00000100:00080000:2.0:1787828520.022250:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828520.022261:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828520.022306:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626847360/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828520.022344:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828520.022369:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828520.022372:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828520.022379:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2405
00000100:00080000:2.0:1787828520.022386:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828520.022392:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828520.022405:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828520.022427:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626847360:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828520.022429:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8b40 x1874671626847488/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828520.022452:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828520.022458:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828520.022461:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828520.022466:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2405
00000100:00080000:2.0:1787828520.022471:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828520.022475:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828520.022481:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828520.022499:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626847616/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828520.022511:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828520.022516:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828520.022519:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828520.022524:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2405
00000100:00080000:2.0:1787828520.022529:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828520.022536:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:3.0:1787828520.022544:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:2.0:1787828520.022575:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828520.022609:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626847744/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828520.022637:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828520/real 1787828520]  req@ffff8cd0c3da8f00 x1874671626847360/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828545 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828520.022678:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626847360/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828545 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828520.022700:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828520.022708:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828520.022716:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626847360:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828520.022765:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626847488:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828520.022817:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828520.022846:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828520/real 1787828520]  req@ffff8cd0c3da8b40 x1874671626847488/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828545 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828520.022871:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8b40 x1874671626847488/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828545 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828520.022911:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828520.022917:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828520.022924:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626847488:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828520.022936:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626847616:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828520.022978:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828520.023003:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828520/real 1787828520]  req@ffff8cd0c3da83c0 x1874671626847616/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828545 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828520.023369:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626847616/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828545 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828520.023389:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828520.023396:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828520.023403:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626847616:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828520.023418:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626847744:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828520.023468:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828520.023494:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828520/real 1787828520]  req@ffff8cd0c3da9680 x1874671626847744/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828545 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828520.023520:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626847744/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828545 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828520.023537:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828520.023544:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828520.023550:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626847744:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828520.214212:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828520.598226:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828521.238248:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828521.622282:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828522.262256:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828522.646280:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828523.286200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828523.670281:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828524.310201:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828524.694189:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828525.142252:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828525.142307:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828525.142314:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828525.142330:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2410
00000100:00080000:3.0:1787828525.142348:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828525.142362:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828525.142418:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626847872/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828525.142468:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828525.142473:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828525.142476:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828525.142482:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2410
00000100:00080000:3.0:1787828525.142489:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828525.142496:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828525.142510:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828525.142524:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626847872:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828525.142533:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626848000/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828525.142551:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828525.142556:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828525.142559:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828525.142563:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2410
00000100:00080000:3.0:1787828525.142567:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828525.142572:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828525.142578:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828525.142595:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626848128/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828525.142619:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:2.0:1787828525.142631:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787828525.142639:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828525.142644:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828525.142652:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2410
00000100:00080000:3.0:1787828525.142661:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828525.142667:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828525.142675:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828525.142693:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828525/real 1787828525]  req@ffff8cd0c8eae1c0 x1874671626847872/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828550 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828525.142699:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626848256/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828525.142757:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626847872/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828550 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828525.142776:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828525.142782:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828525.142788:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626847872:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828525.142819:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626848000:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828525.142881:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828525.142909:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828525/real 1787828525]  req@ffff8cd0c8eac780 x1874671626848000/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828550 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828525.142926:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626848000/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828550 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828525.142937:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828525.142941:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828525.142945:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626848000:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828525.142952:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626848128:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828525.142994:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828525.143008:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828525/real 1787828525]  req@ffff8cd0c8eada40 x1874671626848128/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828550 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828525.143023:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626848128/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828550 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828525.143033:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828525.143036:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828525.143040:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626848128:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828525.143047:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626848256:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828525.143067:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828525.143078:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828525/real 1787828525]  req@ffff8cd0c8eaf840 x1874671626848256/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828550 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828525.143124:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626848256/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828550 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828525.143134:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828525.143138:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828525.143142:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626848256:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828525.334216:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828525.718246:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828526.358253:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828526.742218:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828527.382276:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828527.766366:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828528.406295:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828528.790306:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828529.430220:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828529.814319:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828530.262215:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828530.262228:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828530.262234:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828530.262246:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2416
00000100:00080000:3.0:1787828530.262271:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828530.262280:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828530.262332:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626848384/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828530.262387:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828530.262391:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828530.262393:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828530.262398:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2416
00000100:00080000:3.0:1787828530.262402:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828530.262406:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828530.262418:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828530.262431:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626848512/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828530.262442:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828530.262446:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828530.262448:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828530.262452:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2416
00000100:00080000:3.0:1787828530.262458:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828530.262464:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828530.262472:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828530.262503:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626848384:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828530.262507:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626848640/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828530.262523:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828530.262527:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828530.262529:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828530.262534:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2416
00000100:00080000:3.0:1787828530.262545:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828530.262549:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828530.262554:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828530.262565:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626848768/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787828530.262585:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828530.262630:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828530/real 1787828530]  req@ffff8cd0c8eaf0c0 x1874671626848384/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828555 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828530.262659:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626848384/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828555 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828530.262685:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828530.262690:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828530.262695:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626848384:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828530.262703:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626848512:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828530.262736:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828530.262770:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828530/real 1787828530]  req@ffff8cd0c8eac3c0 x1874671626848512/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828555 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828530.262785:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626848512/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828555 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828530.262796:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828530.262800:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828530.262832:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626848512:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828530.262892:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626848640:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828530.262920:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828530.262928:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828530/real 1787828530]  req@ffff8cd0c8eaf840 x1874671626848640/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828555 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828530.262937:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626848640/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828555 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828530.262953:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828530.262956:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828530.262958:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626848640:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828530.262963:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626848768:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828530.262977:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828530.262985:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828530/real 1787828530]  req@ffff8cd0c8eada40 x1874671626848768/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828555 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828530.262993:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626848768/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828555 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828530.262999:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828530.263001:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828530.263003:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626848768:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828530.454208:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828530.838346:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828531.478192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828531.862270:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828532.502284:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828532.886242:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828533.526196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828533.910231:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828534.550178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828534.934291:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828535.382151:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828535.382159:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828535.382162:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828535.382171:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2421
00000100:00080000:3.0:1787828535.382180:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828535.382186:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828535.382219:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626848896/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828535.382240:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828535.382243:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828535.382245:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828535.382247:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2421
00000100:00080000:3.0:1787828535.382250:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828535.382252:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828535.382274:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828535.382286:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626849024/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828535.382287:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626848896:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787828535.382302:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828535.382305:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828535.382307:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828535.382310:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2421
00000100:00080000:3.0:1787828535.382320:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828535.382324:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828535.382328:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828535.382336:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626849152/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828535.382343:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828535.382347:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000800:00000100:2.0:1787828535.382351:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787828535.382351:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828535.382358:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2421
00000100:00080000:3.0:1787828535.382362:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828535.382365:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828535.382370:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828535.382378:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626849280/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828535.382394:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828535/real 1787828535]  req@ffff8cd0c8eac780 x1874671626848896/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828560 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828535.382409:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626848896/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828560 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828535.382417:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828535.382419:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828535.382423:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626848896:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828535.382449:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626849024:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828535.382466:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828535.382475:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828535/real 1787828535]  req@ffff8cd0c8eae1c0 x1874671626849024/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828560 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828535.382483:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626849024/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828560 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828535.382489:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828535.382491:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828535.382493:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626849024:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828535.382497:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626849152:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828535.382508:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828535.382513:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828535/real 1787828535]  req@ffff8cd0c8ead2c0 x1874671626849152/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828560 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828535.382521:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626849152/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828560 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828535.382525:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828535.382527:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828535.382529:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626849152:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828535.382532:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626849280:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828535.382541:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828535.382547:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828535/real 1787828535]  req@ffff8cd0c8eae940 x1874671626849280/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828560 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828535.382563:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626849280/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828560 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828535.382571:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828535.382575:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828535.382578:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626849280:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828535.574155:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828535.958386:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828536.598213:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828536.982279:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828537.622263:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828538.006291:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828538.646146:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828539.030171:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828539.670245:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828540.054430:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828540.502260:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828540.502273:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828540.502278:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828540.502291:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2426
00000100:00080000:1.0:1787828540.502304:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828540.502313:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828540.502371:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626849408/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828540.502432:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828540.502439:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828540.502461:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828540.502469:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2426
00000100:00080000:1.0:1787828540.502475:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828540.502482:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828540.502495:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828540.502516:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626849536/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828540.502527:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626849408:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787828540.502536:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828540.502542:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787828540.502545:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828540.502551:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2426
00000100:00080000:1.0:1787828540.502567:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828540.502574:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828540.502581:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828540.502600:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626849664/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828540.502622:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828540.502626:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828540.502628:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828540.502633:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2426
00000100:00080000:1.0:1787828540.502637:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828540.502641:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828540.502646:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828540.502671:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626849792/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787828540.502676:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828540.502808:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828540/real 1787828540]  req@ffff8cd0d48f4000 x1874671626849408/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828565 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828540.502834:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626849408/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828565 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828540.502848:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828540.502853:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828540.502858:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626849408:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828540.502947:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626849536:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828540.502997:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828540.503025:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828540/real 1787828540]  req@ffff8cd0d48f7480 x1874671626849536/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828565 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828540.503060:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626849536/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828565 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828540.503172:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828540.503177:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828540.503181:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626849536:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828540.503189:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626849664:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828540.503219:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828540.503233:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828540/real 1787828540]  req@ffff8cd0d48f6580 x1874671626849664/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828565 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828540.503250:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626849664/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828565 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828540.503263:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828540.503268:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828540.503274:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626849664:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828540.503285:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626849792:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828540.503332:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828540.503349:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828540/real 1787828540]  req@ffff8cd0d48f4b40 x1874671626849792/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828565 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828540.503362:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626849792/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828565 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828540.503371:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828540.503375:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828540.503387:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626849792:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828540.694181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828541.078240:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828541.718168:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828542.102246:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828542.742278:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828543.126209:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828543.766215:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828544.150362:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828544.790171:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828545.174306:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828545.622312:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828545.622328:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828545.622335:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828545.622352:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2431
00000100:00080000:2.0:1787828545.622370:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828545.622381:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828545.622450:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626849920/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828545.622498:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828545.622503:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828545.622506:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828545.622511:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2431
00000100:00080000:2.0:1787828545.622515:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828545.622521:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828545.622536:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828545.622554:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626850048/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828545.622571:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828545.622576:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828545.622580:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1787828545.622586:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626849920:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828545.622588:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2431
00000100:00080000:2.0:1787828545.622601:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828545.622637:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828545.622646:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828545.622666:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8b40 x1874671626850176/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828545.622680:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828545.622685:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828545.622688:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828545.622693:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2431
00000100:00080000:2.0:1787828545.622723:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828545.622730:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828545.622737:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787828545.622750:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828545.622758:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626850304/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828545.622833:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828545/real 1787828545]  req@ffff8cd0c3da9680 x1874671626849920/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828570 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828545.622862:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626849920/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828570 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828545.622878:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828545.622883:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828545.622889:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626849920:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828545.622924:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626850048:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828545.622958:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828545.622974:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828545/real 1787828545]  req@ffff8cd0c3da83c0 x1874671626850048/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828570 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828545.623015:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626850048/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828570 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828545.623026:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828545.623030:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828545.623035:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626850048:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828545.623042:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626850176:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828545.623073:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828545.623130:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828545/real 1787828545]  req@ffff8cd0c3da8b40 x1874671626850176/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828570 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828545.623148:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8b40 x1874671626850176/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828570 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828545.623158:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828545.623163:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828545.623167:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626850176:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828545.623175:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626850304:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828545.623201:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828545.623214:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828545/real 1787828545]  req@ffff8cd0c3da8f00 x1874671626850304/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828570 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828545.623245:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626850304/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828570 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828545.623275:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828545.623279:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828545.623283:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626850304:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828545.814206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828546.198309:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828546.838168:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828547.222309:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828547.862198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828548.246395:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828548.886267:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828549.270273:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828549.910193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828550.294307:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828550.742147:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828550.742156:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828550.742160:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828550.742170:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2436
00000100:00080000:2.0:1787828550.742181:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828550.742187:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828550.742222:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626850432/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828550.742246:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828550.742249:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828550.742261:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828550.742266:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2436
00000100:00080000:2.0:1787828550.742281:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828550.742285:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828550.742294:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828550.742297:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626850432:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828550.742307:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626850560/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828550.742319:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828550.742322:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828550.742323:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828550.742326:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2436
00000100:00080000:2.0:1787828550.742329:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828550.742332:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828550.742336:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828550.742350:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626850688/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828550.742360:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787828550.742370:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828550.742375:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828550.742378:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828550.742385:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2436
00000100:00080000:2.0:1787828550.742391:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828550.742396:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828550.742401:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828550/real 1787828550]  req@ffff8cd0c3daa940 x1874671626850432/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828575 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1787828550.742405:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828550.742418:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626850432/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828575 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828550.742428:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626850816/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828550.742430:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828550.742440:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828550.742447:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626850432:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828550.742481:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626850560:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828550.742498:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828550.742508:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828550/real 1787828550]  req@ffff8cd0c3da8780 x1874671626850560/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828575 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828550.742525:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626850560/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828575 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828550.742532:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828550.742534:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828550.742537:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626850560:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828550.742542:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626850688:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828550.742569:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828550.742576:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828550/real 1787828550]  req@ffff8cd0c3da9a40 x1874671626850688/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828575 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828550.742585:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626850688/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828575 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828550.742590:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828550.742592:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828550.742594:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626850688:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828550.742598:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626850816:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828550.742610:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828550.742616:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828550/real 1787828550]  req@ffff8cd0c3dab0c0 x1874671626850816/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828575 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828550.742625:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626850816/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828575 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828550.742640:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828550.742654:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828550.742656:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626850816:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828550.934307:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828551.318234:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828551.958194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828552.342339:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828552.982276:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828553.366233:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828554.006211:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828554.390349:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828555.030235:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828555.414226:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828555.862156:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828555.862164:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828555.862168:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828555.862176:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2441
00000100:00080000:3.0:1787828555.862184:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828555.862191:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828555.862223:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626850944/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828555.862258:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828555.862261:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828555.862262:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828555.862265:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2441
00000100:00080000:3.0:1787828555.862268:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828555.862271:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828555.862291:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626850944:192.168.203.145@tcp:250:kworker.0
00000008:00100000:3.0:1787828555.862291:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828555.862306:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626851072/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828555.862315:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828555.862318:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828555.862320:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828555.862323:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2441
00000100:00080000:3.0:1787828555.862326:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828555.862328:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828555.862333:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828555.862340:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626851200/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828555.862347:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828555.862350:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000800:00000100:2.0:1787828555.862354:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787828555.862354:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828555.862361:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2441
00000100:00080000:3.0:1787828555.862366:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828555.862370:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828555.862376:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828555.862388:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626851328/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828555.862394:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828555/real 1787828555]  req@ffff8cd0c8eac000 x1874671626850944/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828555.862428:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626850944/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828555.862437:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828555.862440:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828555.862443:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626850944:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828555.862461:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626851072:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828555.862477:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828555.862486:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828555/real 1787828555]  req@ffff8cd0c8eacf00 x1874671626851072/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828555.862494:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626851072/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828555.862499:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828555.862501:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828555.862503:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626851072:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828555.862507:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626851200:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828555.862529:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828555.862536:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828555/real 1787828555]  req@ffff8cd0c8eae940 x1874671626851200/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828555.862544:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626851200/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828555.862549:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828555.862551:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828555.862552:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626851200:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828555.862566:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626851328:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828555.862577:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828555.862583:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828555/real 1787828555]  req@ffff8cd0c8ead2c0 x1874671626851328/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828555.862590:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626851328/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828555.862595:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828555.862596:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828555.862598:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626851328:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828556.054157:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828556.438311:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828557.078192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828557.462197:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828558.102263:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828558.486317:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828559.126215:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828559.510234:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828560.150220:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828560.534354:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828560.982298:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828560.982314:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828560.982322:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828560.982348:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2446
00000100:00080000:2.0:1787828560.982391:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828560.982425:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828560.982507:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626851456/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828560.982548:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828560.982553:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828560.982555:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828560.982560:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2446
00000100:00080000:2.0:1787828560.982567:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828560.982573:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828560.982586:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828560.982643:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626851584/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828560.982659:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787828560.982664:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626851456:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828560.982665:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828560.982671:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828560.982703:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2446
00000100:00080000:2.0:1787828560.982708:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828560.982713:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828560.982721:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828560.982741:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626851712/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828560.982756:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828560.982761:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828560.982774:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828560.982780:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2446
00000800:00000100:3.0:1787828560.982793:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787828560.982799:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828560.982822:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828560.982832:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828560.982853:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626851840/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828560.982879:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828560/real 1787828560]  req@ffff8cd0c3dabc00 x1874671626851456/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828585 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828560.982915:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626851456/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828585 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828560.982944:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828560.982951:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828560.982959:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626851456:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828560.983016:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626851584:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828560.983065:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828560.983145:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828560/real 1787828560]  req@ffff8cd0c3daa580 x1874671626851584/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828585 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828560.983175:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626851584/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828585 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828560.983189:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828560.983194:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828560.983200:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626851584:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828560.983254:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626851712:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828560.983301:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828560.983355:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828560/real 1787828560]  req@ffff8cd0c3dab840 x1874671626851712/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828585 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828560.983394:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626851712/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828585 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828560.983405:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828560.983408:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828560.983413:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626851712:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828560.983420:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626851840:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828560.983480:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828560.983496:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828560/real 1787828560]  req@ffff8cd0c3da8000 x1874671626851840/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828585 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828560.983510:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626851840/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828585 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828560.983521:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828560.983525:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828560.983529:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626851840:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828561.174215:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828561.558202:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828562.198173:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828562.582296:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828563.222199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828563.606187:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828564.246235:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828564.630344:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828565.270232:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828565.654229:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828566.102225:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828566.102236:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828566.102241:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828566.102252:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2451
00000100:00080000:2.0:1787828566.102280:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828566.102288:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828566.102358:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626851968/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828566.102389:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828566.102392:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828566.102394:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828566.102397:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2451
00000100:00080000:2.0:1787828566.102401:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828566.102414:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828566.102423:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828566.102435:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626852096/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828566.102469:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828566.102472:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828566.102474:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828566.102481:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2451
00000100:00100000:3.0:1787828566.102485:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626851968:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828566.102487:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828566.102498:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828566.102504:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828566.102521:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626852224/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828566.102528:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828566.102532:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828566.102534:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828566.102539:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2451
00000100:00080000:2.0:1787828566.102549:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828566.102569:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:3.0:1787828566.102574:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:2.0:1787828566.102608:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828566.102629:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daad00 x1874671626852352/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828566.102651:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828566/real 1787828566]  req@ffff8cd0c3da9e00 x1874671626851968/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828591 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828566.102669:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626851968/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828591 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828566.102678:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828566.102681:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828566.102684:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626851968:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828566.102704:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626852096:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828566.102726:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828566.102737:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828566/real 1787828566]  req@ffff8cd0c3da92c0 x1874671626852096/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828591 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828566.102747:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626852096/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828591 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828566.102753:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828566.102756:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828566.102758:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626852096:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828566.102763:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626852224:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828566.102792:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828566.102799:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828566/real 1787828566]  req@ffff8cd0c3daa1c0 x1874671626852224/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828591 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828566.102812:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626852224/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828591 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828566.102817:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828566.102820:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828566.102822:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626852224:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828566.102826:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626852352:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828566.102837:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828566.102843:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828566/real 1787828566]  req@ffff8cd0c3daad00 x1874671626852352/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828591 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828566.102850:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daad00 x1874671626852352/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828591 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828566.102855:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828566.102857:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828566.102858:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626852352:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828566.294142:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828566.678232:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828567.318154:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828567.702195:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828568.342168:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828568.726255:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828569.366192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828569.750276:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828570.390159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828570.774376:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828571.222300:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828571.222310:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828571.222315:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828571.222325:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2457
00000100:00080000:0.0:1787828571.222359:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828571.222367:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828571.222405:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626852480/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828571.222437:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828571.222441:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828571.222443:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828571.222446:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2457
00000100:00080000:0.0:1787828571.222449:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828571.222453:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828571.222463:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828571.222475:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626852608/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828571.222488:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828571.222491:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828571.222493:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828571.222499:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2457
00000100:00100000:3.0:1787828571.222501:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626852480:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828571.222516:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828571.222538:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828571.222544:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828571.222576:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626852736/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828571.222588:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828571.222600:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828571.222602:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828571.222607:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2457
00000800:00000100:3.0:1787828571.222633:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787828571.222637:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828571.222648:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828571.222658:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828571.222684:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626852864/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828571.222701:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828571/real 1787828571]  req@ffff8cd0c8e9e940 x1874671626852480/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828596 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828571.222744:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626852480/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828596 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828571.222758:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828571.222763:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828571.222769:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626852480:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828571.222797:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626852608:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828571.222838:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828571.222850:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828571/real 1787828571]  req@ffff8cd0c8e9de00 x1874671626852608/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828596 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828571.222864:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626852608/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828596 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828571.222876:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828571.222881:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828571.222886:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626852608:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828571.222895:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626852736:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828571.222925:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828571.222953:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828571/real 1787828571]  req@ffff8cd0c8e9cf00 x1874671626852736/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828596 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828571.222965:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626852736/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828596 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828571.222973:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828571.222976:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828571.222979:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626852736:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828571.222985:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626852864:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828571.223004:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828571.223013:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828571/real 1787828571]  req@ffff8cd0c8e9fc00 x1874671626852864/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828596 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828571.223024:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626852864/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828596 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828571.223030:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828571.223033:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828571.223035:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626852864:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828571.414136:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828571.798287:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828572.438154:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828572.822314:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828573.462193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828573.846240:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828574.486180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828574.870280:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828575.510170:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828575.894251:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828576.342184:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828576.342195:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828576.342200:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828576.342212:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2462
00000100:00080000:1.0:1787828576.342227:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828576.342237:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828576.342285:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626852992/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828576.342318:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828576.342323:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828576.342325:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828576.342331:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2462
00000100:00080000:1.0:1787828576.342337:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828576.342345:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828576.342359:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828576.342373:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626852992:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787828576.342381:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626853120/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828576.342396:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828576.342402:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787828576.342404:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828576.342407:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2462
00000100:00080000:1.0:1787828576.342412:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828576.342426:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828576.342433:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828576.342447:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626853248/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828576.342458:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828576.342461:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828576.342463:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:3.0:1787828576.342471:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787828576.342480:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2462
00000100:00080000:1.0:1787828576.342506:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828576.342512:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828576.342518:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828576.342534:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626853376/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828576.342539:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828576/real 1787828576]  req@ffff8cd0d48f7840 x1874671626852992/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828601 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828576.342573:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626852992/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828601 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828576.342584:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828576.342589:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828576.342593:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626852992:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828576.342616:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626853120:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828576.342639:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828576.342651:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828576/real 1787828576]  req@ffff8cd0d48f5a40 x1874671626853120/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828601 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828576.342663:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626853120/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828601 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828576.342671:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828576.342674:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828576.342677:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626853120:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828576.342682:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626853248:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828576.342698:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828576.342706:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828576/real 1787828576]  req@ffff8cd0d48f43c0 x1874671626853248/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828601 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828576.342732:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626853248/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828601 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828576.342739:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828576.342742:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828576.342745:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626853248:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828576.342750:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626853376:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828576.342768:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828576.342777:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828576/real 1787828576]  req@ffff8cd0d48f70c0 x1874671626853376/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828601 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828576.342787:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626853376/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828601 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828576.342794:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828576.342796:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828576.342799:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626853376:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828576.534266:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828576.918234:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828577.558181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828577.942238:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828578.582192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828578.966298:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828579.606175:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828579.990225:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828580.630194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828581.014390:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828581.462198:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828581.462209:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828581.462214:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828581.462226:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2467
00000100:00080000:1.0:1787828581.462240:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828581.462248:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828581.462299:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626853504/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828581.462326:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828581.462330:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828581.462352:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828581.462357:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2467
00000100:00080000:1.0:1787828581.462364:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1787828581.462364:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626853504:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787828581.462370:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828581.462382:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828581.462399:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626853632/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828581.462409:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828581.462413:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787828581.462415:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828581.462418:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2467
00000100:00080000:1.0:1787828581.462421:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828581.462424:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828581.462428:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828581.462436:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626853760/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828581.462444:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828581.462463:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828581.462465:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828581.462469:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2467
00000100:00080000:1.0:1787828581.462473:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828581.462476:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828581.462480:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828581.462523:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626853888/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828581.462544:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626853632:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828581.462581:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626853760:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828581.462604:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626853888:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828581.654185:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828582.038228:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828582.678192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828583.062296:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828583.702229:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828584.086200:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:2.0:1787828584.534473:0:2346:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:2.0:1787828584.534533:0:2346:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:2.0:1787828584.534555:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787828584.534604:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787828584.534639:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787828584.534647:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:3.0:1787828584.534659:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828581/real 1787828584]  req@ffff8cd0d48f7c00 x1874671626853632/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828606 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828584.534686:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626853632/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828606 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828584.534698:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828584.534702:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828584.534706:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626853632:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828584.534714:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828581/real 1787828584]  req@ffff8cd0d48f52c0 x1874671626853760/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828606 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828584.534726:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626853760/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828606 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828584.534736:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828584.534739:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828584.534742:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626853760:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828584.534762:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828581/real 1787828584]  req@ffff8cd0d48f6d00 x1874671626853888/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828606 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828584.534776:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626853888/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828606 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828584.534784:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828584.534788:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828584.534792:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626853888:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828584.534800:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828581/real 1787828584]  req@ffff8cd0d48f61c0 x1874671626853504/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828606 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828584.534814:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626853504/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828606 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828584.534822:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828584.534827:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828584.534830:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626853504:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828584.726203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828585.110259:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828585.750181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828586.134229:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828586.582208:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828586.582216:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828586.582220:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828586.582229:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2472
00000100:00080000:3.0:1787828586.582238:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828586.582245:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828586.582288:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626854016/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828586.582313:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828586.582317:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828586.582318:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828586.582321:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2472
00000100:00080000:3.0:1787828586.582325:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828586.582329:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828586.582337:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828586.582346:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626854144/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828586.582351:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626854016:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787828586.582369:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828586.582373:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828586.582376:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828586.582381:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2472
00000100:00080000:3.0:1787828586.582386:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828586.582390:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828586.582397:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828586.582428:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626854272/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787828586.582431:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787828586.582447:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828586.582452:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828586.582454:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828586.582465:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2472
00000100:00080000:3.0:1787828586.582469:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828586.582473:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828586.582478:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828586.582479:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828586/real 1787828586]  req@ffff8cd0c8eae1c0 x1874671626854016/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828611 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828586.582495:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626854400/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828586.582502:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626854016/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828611 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828586.582530:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828586.582535:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828586.582551:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626854016:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828586.582580:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626854144:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828586.582607:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828586.582622:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828586/real 1787828586]  req@ffff8cd0c8eac780 x1874671626854144/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828611 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828586.582640:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626854144/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828611 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828586.582648:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828586.582651:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828586.582654:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626854144:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828586.582661:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626854272:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828586.582685:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828586.582708:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828586/real 1787828586]  req@ffff8cd0c8eada40 x1874671626854272/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828611 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828586.582719:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626854272/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828611 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828586.582727:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828586.582729:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828586.582732:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626854272:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828586.582738:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626854400:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828586.582760:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828586.582772:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828586/real 1787828586]  req@ffff8cd0c8eaf840 x1874671626854400/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828611 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828586.582783:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626854400/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828611 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828586.582791:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828586.582794:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828586.582798:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626854400:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828586.774177:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828587.158209:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828587.798171:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828588.182214:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828588.822233:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828589.206337:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828589.846172:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828590.230191:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828590.870151:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828591.254228:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828591.702155:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828591.702163:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828591.702167:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828591.702175:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2477
00000100:00080000:3.0:1787828591.702185:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828591.702190:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828591.702222:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626854528/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828591.702244:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828591.702247:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828591.702252:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828591.702255:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2477
00000100:00080000:3.0:1787828591.702272:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828591.702276:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828591.702284:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828591.702298:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626854528:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828591.702306:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626854656/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828591.702316:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828591.702320:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828591.702322:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828591.702327:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2477
00000100:00080000:3.0:1787828591.702331:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828591.702335:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828591.702341:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828591.702354:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626854784/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787828591.702361:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787828591.702371:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828591.702393:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828591.702396:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828591.702401:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2477
00000100:00080000:3.0:1787828591.702406:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828591.702410:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828591.702416:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828591.702441:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626854912/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828591.702444:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828591/real 1787828591]  req@ffff8cd0c8eac3c0 x1874671626854528/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828616 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828591.702472:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626854528/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828616 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828591.702489:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828591.702493:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828591.702496:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626854528:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828591.702517:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626854656:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828591.702537:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828591.702547:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828591/real 1787828591]  req@ffff8cd0c8eaf0c0 x1874671626854656/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828616 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828591.702555:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626854656/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828616 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828591.702560:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828591.702563:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828591.702565:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626854656:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828591.702569:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626854784:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828591.702580:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828591.702585:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828591/real 1787828591]  req@ffff8cd0c8eaf840 x1874671626854784/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828616 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828591.702593:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626854784/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828616 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828591.702598:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828591.702600:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828591.702601:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626854784:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828591.702611:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626854912:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828591.702625:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828591.702631:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828591/real 1787828591]  req@ffff8cd0c8eada40 x1874671626854912/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828616 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828591.702639:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626854912/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828616 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828591.702644:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828591.702645:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828591.702647:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626854912:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828591.894189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828592.278216:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828592.918171:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828593.302268:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828593.942156:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828594.326181:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828594.966165:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828595.350253:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828595.990211:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828596.374222:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828596.822218:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828596.822228:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828596.822232:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828596.822243:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2482
00000100:00080000:3.0:1787828596.822255:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828596.822262:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828596.822305:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626855040/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828596.822332:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828596.822335:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828596.822336:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828596.822350:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2482
00000100:00080000:3.0:1787828596.822353:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828596.822357:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828596.822367:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828596.822380:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626855168/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828596.822389:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626855040:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787828596.822391:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828596.822424:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828596.822425:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828596.822429:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2482
00000100:00080000:3.0:1787828596.822433:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828596.822437:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828596.822442:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828596.822451:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626855296/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828596.822460:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828596.822463:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828596.822465:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1787828596.822471:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787828596.822471:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2482
00000100:00080000:3.0:1787828596.822474:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828596.822479:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828596.822482:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828596.822491:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626855424/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828596.822526:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828596/real 1787828596]  req@ffff8cd0c8eac780 x1874671626855040/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828621 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828596.822550:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626855040/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828621 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828596.822560:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828596.822563:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828596.822566:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626855040:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828596.822591:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626855168:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828596.822612:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828596.822621:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828596/real 1787828596]  req@ffff8cd0c8eae1c0 x1874671626855168/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828621 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828596.822638:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626855168/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828621 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828596.822644:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828596.822646:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828596.822649:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626855168:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828596.822652:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626855296:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828596.822668:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828596.822675:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828596/real 1787828596]  req@ffff8cd0c8ead2c0 x1874671626855296/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828621 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828596.822683:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626855296/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828621 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828596.822697:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828596.822699:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828596.822701:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626855296:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828596.822705:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626855424:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828596.822717:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828596.822724:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828596/real 1787828596]  req@ffff8cd0c8eae940 x1874671626855424/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828621 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828596.822732:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626855424/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828621 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828596.822747:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828596.822749:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828596.822752:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626855424:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828597.014180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828597.398235:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828598.038193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828598.422172:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828599.062171:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828599.446225:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828600.086169:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828600.470189:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828601.110198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828601.494273:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828601.942209:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828601.942218:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828601.942221:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828601.942230:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2487
00000100:00080000:2.0:1787828601.942240:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828601.942245:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828601.942279:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daad00 x1874671626855552/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828601.942323:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828601.942326:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828601.942327:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828601.942331:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2487
00000100:00080000:2.0:1787828601.942334:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828601.942337:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828601.942346:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828601.942356:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626855680/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828601.942364:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828601.942367:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:3.0:1787828601.942380:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626855552:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828601.942385:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828601.942398:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2487
00000100:00080000:2.0:1787828601.942402:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828601.942404:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828601.942410:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828601.942421:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626855808/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828601.942428:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828601.942430:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828601.942433:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828601.942435:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2487
00000100:00080000:2.0:1787828601.942439:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828601.942442:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828601.942448:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828601.942456:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626855936/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828601.942479:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828601.942527:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828601/real 1787828601]  req@ffff8cd0c3daad00 x1874671626855552/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828626 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828601.942543:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daad00 x1874671626855552/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828626 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828601.942553:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828601.942556:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828601.942559:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626855552:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828601.942583:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626855680:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828601.942600:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828601.942609:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828601/real 1787828601]  req@ffff8cd0c3daa1c0 x1874671626855680/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828626 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828601.942617:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626855680/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828626 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828601.942622:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828601.942624:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828601.942626:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626855680:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828601.942630:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626855808:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828601.942643:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828601.942649:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828601/real 1787828601]  req@ffff8cd0c3da92c0 x1874671626855808/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828626 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828601.942674:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626855808/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828626 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828601.942679:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828601.942681:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828601.942683:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626855808:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828601.942687:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626855936:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828601.942701:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828601.942708:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828601/real 1787828601]  req@ffff8cd0c3da9e00 x1874671626855936/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828626 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828601.942715:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626855936/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828626 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828601.942720:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828601.942722:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828601.942724:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626855936:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828602.134228:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828602.518187:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828603.158262:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828603.542332:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828604.182168:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828604.566200:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828605.206212:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828605.590294:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828606.230159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828606.614199:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828607.062236:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828607.062248:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828607.062292:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828607.062304:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2492
00000100:00080000:0.0:1787828607.062316:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828607.062323:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828607.062384:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626856064/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828607.062412:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828607.062415:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828607.062417:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828607.062420:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2492
00000100:00080000:0.0:1787828607.062423:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828607.062428:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828607.062440:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828607.062457:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626856192/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828607.062470:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828607.062474:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828607.062477:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828607.062481:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2492
00000100:00080000:0.0:1787828607.062485:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828607.062489:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828607.062494:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828607.062505:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626856320/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828607.062514:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626856064:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828607.062531:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828607.062536:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828607.062538:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828607.062552:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2492
00000100:00080000:0.0:1787828607.062558:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828607.062563:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828607.062580:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828607.062592:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626856448/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828607.062668:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828607.062910:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828607/real 1787828607]  req@ffff8cd0c8e9e580 x1874671626856064/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828632 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828607.062943:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626856064/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828632 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828607.062953:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828607.062957:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828607.062960:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626856064:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828607.062966:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626856192:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828607.063045:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828607.063134:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828607/real 1787828607]  req@ffff8cd0c8e9fc00 x1874671626856192/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828632 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828607.063144:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626856192/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828632 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828607.063158:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828607.063160:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828607.063162:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626856192:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828607.063206:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626856320:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828607.063243:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828607.063252:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828607/real 1787828607]  req@ffff8cd0c8e9cf00 x1874671626856320/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828632 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828607.063288:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626856320/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828632 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828607.063295:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828607.063297:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828607.063300:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626856320:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828607.063304:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626856448:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828607.063322:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828607.063330:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828607/real 1787828607]  req@ffff8cd0c8e9de00 x1874671626856448/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828632 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828607.063338:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626856448/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828632 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828607.063343:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828607.063344:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828607.063346:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626856448:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828607.254170:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828607.638325:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828608.278149:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828608.662230:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828609.302130:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828609.686328:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828610.326157:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828610.710174:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828611.350171:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828611.734253:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828612.182285:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828612.182296:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828612.182301:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828612.182311:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2498
00000100:00080000:1.0:1787828612.182323:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828612.182330:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828612.182383:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626856576/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828612.182406:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828612.182408:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828612.182410:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828612.182413:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2498
00000100:00080000:1.0:1787828612.182415:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828612.182418:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828612.182427:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828612.182438:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626856704/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828612.182446:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828612.182450:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787828612.182451:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828612.182454:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2498
00000100:00080000:1.0:1787828612.182456:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1787828612.182467:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626856576:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787828612.182480:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828612.182487:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828612.182497:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626856832/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828612.182507:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828612.182511:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828612.182513:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828612.182517:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2498
00000100:00080000:1.0:1787828612.182523:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828612.182528:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828612.182534:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828612.182547:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626856960/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828612.182561:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828612.182612:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828612/real 1787828612]  req@ffff8cd0d48f61c0 x1874671626856576/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828637 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828612.182632:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626856576/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828637 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828612.182645:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828612.182649:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828612.182654:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626856576:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828612.182663:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626856704:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828612.182684:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828612.182696:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828612/real 1787828612]  req@ffff8cd0d48f6d00 x1874671626856704/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828637 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828612.182709:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626856704/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828637 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828612.182718:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828612.182721:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828612.182725:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626856704:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828612.182752:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626856832:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828612.182779:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828612.182790:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828612/real 1787828612]  req@ffff8cd0d48f52c0 x1874671626856832/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828637 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828612.182816:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626856832/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828637 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828612.182825:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828612.182828:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828612.182831:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626856832:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828612.182837:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626856960:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828612.182857:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828612.182867:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828612/real 1787828612]  req@ffff8cd0d48f7c00 x1874671626856960/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828637 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828612.182880:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626856960/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828637 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828612.182898:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828612.182901:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828612.182905:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626856960:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828612.374155:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828612.758263:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828613.398171:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828613.782278:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828614.422149:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828614.806230:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828615.446228:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828615.830253:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828616.470189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828616.854209:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828617.302180:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828617.302188:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828617.302192:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828617.302211:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2503
00000100:00080000:2.0:1787828617.302220:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828617.302226:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828617.302261:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626857088/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828617.302317:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828617.302320:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828617.302321:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828617.302324:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2503
00000100:00080000:2.0:1787828617.302327:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828617.302331:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828617.302340:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828617.302352:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626857216/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828617.302361:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787828617.302363:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626857088:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828617.302365:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828617.302368:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828617.302372:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2503
00000100:00080000:2.0:1787828617.302380:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828617.302387:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828617.302391:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828617.302405:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626857344/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828617.302417:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828617.302420:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828617.302422:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828617.302427:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2503
00000800:00000100:3.0:1787828617.302439:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787828617.302440:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828617.302449:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828617.302456:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828617.302470:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626857472/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828617.302484:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828617/real 1787828617]  req@ffff8cd0c3da8000 x1874671626857088/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828642 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828617.302504:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626857088/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828642 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828617.302515:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828617.302519:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828617.302523:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626857088:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828617.302545:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626857216:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828617.302567:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828617.302580:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828617/real 1787828617]  req@ffff8cd0c3dab840 x1874671626857216/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828642 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828617.302593:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626857216/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828642 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828617.302602:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828617.302605:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828617.302609:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626857216:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828617.302615:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626857344:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828617.302635:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828617.302646:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828617/real 1787828617]  req@ffff8cd0c3daa580 x1874671626857344/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828642 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828617.302671:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626857344/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828642 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828617.302680:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828617.302683:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828617.302686:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626857344:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828617.302692:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626857472:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828617.302711:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828617.302722:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828617/real 1787828617]  req@ffff8cd0c3dabc00 x1874671626857472/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828642 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828617.302734:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626857472/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828642 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828617.302752:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828617.302755:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828617.302759:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626857472:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828617.494175:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828617.878291:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828618.518150:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828618.902202:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828619.542159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828619.926280:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828620.566167:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828620.950230:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828621.590213:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828621.974253:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828622.422187:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828622.422198:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828622.422202:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828622.422213:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2508
00000100:00080000:1.0:1787828622.422228:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828622.422236:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828622.422284:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626857600/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828622.422312:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828622.422315:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828622.422332:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828622.422336:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2508
00000100:00080000:1.0:1787828622.422341:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828622.422345:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828622.422355:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828622.422367:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626857728/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828622.422372:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626857600:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787828622.422377:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828622.422380:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787828622.422383:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828622.422390:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2508
00000100:00080000:1.0:1787828622.422393:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828622.422395:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828622.422399:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828622.422407:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626857856/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828622.422414:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828622.422417:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828622.422418:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828622.422421:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2508
00000100:00080000:1.0:1787828622.422423:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828622.422425:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828622.422428:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828622.422442:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626857984/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828622.422479:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828622.422531:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828622/real 1787828622]  req@ffff8cd0d48f70c0 x1874671626857600/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828647 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828622.422556:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626857600/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828647 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828622.422568:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828622.422573:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828622.422578:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626857600:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828622.422607:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626857728:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828622.422636:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828622.422652:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828622/real 1787828622]  req@ffff8cd0d48f43c0 x1874671626857728/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828647 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828622.422667:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626857728/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828647 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828622.422676:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828622.422680:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828622.422684:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626857728:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828622.422690:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626857856:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828622.422714:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828622.422727:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828622/real 1787828622]  req@ffff8cd0d48f5a40 x1874671626857856/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828647 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828622.422758:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626857856/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828647 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828622.422767:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828622.422771:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828622.422775:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626857856:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828622.422782:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626857984:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828622.422806:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828622.422818:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828622/real 1787828622]  req@ffff8cd0d48f7840 x1874671626857984/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828647 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828622.422832:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626857984/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828647 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828622.422841:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828622.422845:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828622.422848:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626857984:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828622.614130:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828622.998261:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828623.638144:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828624.022273:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828624.662253:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828625.046268:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828625.686254:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828626.070221:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828626.710184:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828627.094207:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828627.542260:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828627.542268:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828627.542272:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828627.542280:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2513
00000100:00080000:2.0:1787828627.542289:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828627.542295:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828627.542328:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626858112/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828627.542355:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828627.542357:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828627.542359:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828627.542361:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2513
00000100:00080000:2.0:1787828627.542364:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828627.542366:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828627.542374:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828627.542382:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626858240/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828627.542389:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828627.542406:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828627.542407:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828627.542410:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2513
00000100:00080000:2.0:1787828627.542414:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828627.542418:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828627.542422:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626858112:192.168.203.145@tcp:250:kworker.0
00000008:00100000:2.0:1787828627.542423:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828627.542434:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626858368/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828627.542448:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828627.542451:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828627.542452:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828627.542455:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2513
00000100:00080000:2.0:1787828627.542457:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828627.542460:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828627.542463:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828627.542471:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626858496/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828627.542498:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828627.542533:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828627/real 1787828627]  req@ffff8cd0c3dab0c0 x1874671626858112/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828652 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828627.542546:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626858112/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828652 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828627.542554:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828627.542556:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828627.542559:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626858112:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828627.542564:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626858240:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828627.542578:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828627.542586:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828627/real 1787828627]  req@ffff8cd0c3da9a40 x1874671626858240/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828652 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828627.542593:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626858240/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828652 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828627.542598:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828627.542600:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828627.542602:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626858240:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828627.542620:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626858368:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828627.542632:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828627.542638:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828627/real 1787828627]  req@ffff8cd0c3da8780 x1874671626858368/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828652 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828627.542656:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626858368/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828652 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828627.542661:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828627.542663:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828627.542665:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626858368:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828627.542669:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626858496:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828627.542681:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828627.542687:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828627/real 1787828627]  req@ffff8cd0c3daa940 x1874671626858496/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828652 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828627.542694:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626858496/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828652 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828627.542699:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828627.542701:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828627.542703:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626858496:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828627.734189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828628.118278:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828628.758184:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828629.142214:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828629.782257:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828630.166205:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828630.806208:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828631.190334:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828631.830251:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828632.214279:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828632.662168:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828632.662177:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828632.662181:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828632.662189:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2518
00000100:00080000:2.0:1787828632.662198:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828632.662205:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828632.662239:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626858624/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828632.662276:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828632.662279:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828632.662280:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828632.662284:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2518
00000100:00080000:2.0:1787828632.662287:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828632.662290:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828632.662298:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828632.662309:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8b40 x1874671626858752/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828632.662319:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828632.662323:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:3.0:1787828632.662323:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626858624:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828632.662327:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828632.662335:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2518
00000100:00080000:2.0:1787828632.662338:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828632.662340:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828632.662343:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828632.662353:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626858880/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828632.662360:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828632.662363:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828632.662364:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828632.662368:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2518
00000100:00080000:2.0:1787828632.662377:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828632.662381:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828632.662402:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787828632.662405:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828632.662414:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626859008/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828632.662443:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828632/real 1787828632]  req@ffff8cd0c3da8f00 x1874671626858624/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828657 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828632.662466:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626858624/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828657 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828632.662474:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828632.662477:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828632.662480:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626858624:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828632.662498:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626858752:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828632.662517:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828632.662527:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828632/real 1787828632]  req@ffff8cd0c3da8b40 x1874671626858752/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828657 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828632.662535:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8b40 x1874671626858752/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828657 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828632.662550:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828632.662552:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828632.662555:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626858752:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828632.662559:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626858880:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828632.662572:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828632.662579:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828632/real 1787828632]  req@ffff8cd0c3da83c0 x1874671626858880/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828657 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828632.662587:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626858880/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828657 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828632.662601:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828632.662604:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828632.662606:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626858880:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828632.662610:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626859008:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828632.662624:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828632.662647:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828632/real 1787828632]  req@ffff8cd0c3da9680 x1874671626859008/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828657 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828632.662654:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626859008/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828657 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828632.662660:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828632.662662:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828632.662664:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626859008:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828632.854152:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828633.238221:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828633.878145:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828634.262203:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828634.902176:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828635.286268:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828635.926168:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828636.310223:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828636.950144:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828637.334282:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828637.782248:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828637.782259:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828637.782263:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828637.782276:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2523
00000100:00080000:3.0:1787828637.782291:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828637.782301:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828637.782382:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626859136/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828637.782415:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828637.782419:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828637.782422:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828637.782428:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2523
00000100:00080000:3.0:1787828637.782433:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828637.782440:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828637.782461:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828637.782477:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626859136:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828637.782477:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626859264/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828637.782502:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828637.782506:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828637.782508:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828637.782513:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2523
00000100:00080000:3.0:1787828637.782516:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828637.782530:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828637.782534:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828637.782549:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626859392/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828637.782556:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828637.782560:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828637.782561:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828637.782571:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2523
00000100:00080000:3.0:1787828637.782578:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828637.782582:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828637.782586:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787828637.782596:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828637.782598:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626859520/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828637.782658:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828637/real 1787828637]  req@ffff8cd0c8eacf00 x1874671626859136/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828637.782678:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626859136/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828637.782704:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828637.782709:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828637.782715:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626859136:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828637.782746:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626859264:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828637.782779:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828637.782796:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828637/real 1787828637]  req@ffff8cd0c8eac000 x1874671626859264/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828637.782814:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626859264/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828637.782826:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828637.782831:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828637.782847:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626859264:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828637.782853:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626859392:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828637.782876:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828637.782889:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828637/real 1787828637]  req@ffff8cd0c8eae940 x1874671626859392/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828637.782922:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626859392/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828637.782934:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828637.782939:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828637.782945:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626859392:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828637.782955:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626859520:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828637.782990:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828637.783002:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828637/real 1787828637]  req@ffff8cd0c8ead2c0 x1874671626859520/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828637.783014:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626859520/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828637.783021:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828637.783024:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828637.783028:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626859520:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828637.974156:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828638.358199:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828638.998149:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828639.382188:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828640.022157:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828640.406177:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828641.046185:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828641.430197:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828642.070190:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828642.454246:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828642.902286:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828642.902298:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828642.902303:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828642.902336:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2528
00000100:00080000:0.0:1787828642.902348:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828642.902356:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828642.902410:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626859648/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828642.902438:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828642.902443:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828642.902445:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828642.902450:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2528
00000100:00080000:0.0:1787828642.902455:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828642.902460:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828642.902475:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828642.902499:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626859648:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787828642.902500:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626859776/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828642.902532:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828642.902536:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828642.902557:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828642.902563:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2528
00000100:00080000:0.0:1787828642.902567:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828642.902571:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828642.902577:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828642.902589:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626859904/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828642.902597:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:2.0:1787828642.902608:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787828642.902609:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828642.902611:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828642.902616:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2528
00000100:00080000:0.0:1787828642.902619:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828642.902624:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828642.902629:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828642.902643:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626860032/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828642.902677:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828642/real 1787828642]  req@ffff8cd0c8e9e940 x1874671626859648/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828642.902709:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626859648/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828642.902722:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828642.902725:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828642.902728:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626859648:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828642.902751:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626859776:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828642.902773:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828642.902782:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828642/real 1787828642]  req@ffff8cd0c8e9d680 x1874671626859776/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828642.902791:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626859776/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828642.902797:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828642.902799:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828642.902801:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626859776:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828642.902805:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626859904:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828642.902818:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828642.902824:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828642/real 1787828642]  req@ffff8cd0c8e9c3c0 x1874671626859904/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828642.902831:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626859904/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828642.902846:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828642.902849:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828642.902851:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626859904:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828642.902856:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626860032:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828642.902870:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828642.902877:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828642/real 1787828642]  req@ffff8cd0c8e9ed00 x1874671626860032/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828642.902884:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626860032/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828642.902889:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828642.902891:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828642.902909:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626860032:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828643.094223:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828643.478238:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828644.118168:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828644.502244:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828645.142214:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828645.526269:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828646.166176:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828646.550208:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828647.190188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828647.574171:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828648.022196:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828648.022204:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828648.022207:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828648.022215:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2533
00000100:00080000:3.0:1787828648.022225:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828648.022231:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828648.022263:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626860160/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828648.022285:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828648.022288:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828648.022302:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828648.022306:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2533
00000100:00080000:3.0:1787828648.022309:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828648.022313:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828648.022322:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828648.022334:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626860288/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828648.022335:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626860160:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787828648.022350:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828648.022352:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828648.022354:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828648.022357:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2533
00000100:00080000:3.0:1787828648.022359:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828648.022361:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828648.022365:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828648.022374:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626860416/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828648.022380:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828648.022382:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828648.022384:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828648.022386:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2533
00000100:00080000:3.0:1787828648.022389:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828648.022391:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828648.022394:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828648.022406:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626860544/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828648.022443:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626860288:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828648.022472:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626860416:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828648.022492:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626860544:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828648.214141:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828648.598208:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828649.238179:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828649.622280:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828650.262193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828650.646224:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:2.0:1787828651.094395:0:2343:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:2.0:1787828651.094434:0:2343:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:2.0:1787828651.094449:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787828651.094493:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787828651.094513:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787828651.094530:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:3.0:1787828651.094543:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828648/real 1787828651]  req@ffff8cd0c8eac780 x1874671626860288/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828673 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828651.094578:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626860288/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828673 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828651.094590:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828651.094595:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828651.094599:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626860288:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828651.094608:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828648/real 1787828651]  req@ffff8cd0c8eada40 x1874671626860416/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828673 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828651.094622:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626860416/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828673 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828651.094631:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828651.094633:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828651.094636:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626860416:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828651.094640:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828648/real 1787828651]  req@ffff8cd0c8eaf840 x1874671626860544/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828673 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828651.094652:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626860544/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828673 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828651.094667:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828651.094675:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828651.094678:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626860544:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828651.094685:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828648/real 1787828651]  req@ffff8cd0c8eae1c0 x1874671626860160/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828673 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828651.094699:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626860160/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828673 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828651.094708:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828651.094711:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828651.094714:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626860160:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828651.286132:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828651.670218:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828652.310150:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828652.694255:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828653.142162:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828653.142175:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828653.142179:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828653.142190:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2538
00000100:00080000:3.0:1787828653.142202:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828653.142209:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828653.142248:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626860672/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828653.142276:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828653.142279:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828653.142297:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828653.142301:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2538
00000100:00080000:3.0:1787828653.142305:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828653.142308:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828653.142318:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828653.142329:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626860800/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828653.142338:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828653.142341:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828653.142342:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828653.142346:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2538
00000100:00080000:3.0:1787828653.142349:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828653.142352:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828653.142355:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828653.142370:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626860928/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828653.142371:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626860672:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787828653.142399:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828653.142403:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828653.142406:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828653.142412:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2538
00000100:00080000:3.0:1787828653.142416:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828653.142421:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828653.142429:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787828653.142461:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828653.142468:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626861056/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828653.142530:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828653/real 1787828653]  req@ffff8cd0c8eae1c0 x1874671626860672/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828678 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828653.142564:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626860672/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828678 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828653.142577:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828653.142581:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828653.142585:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626860672:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828653.142596:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626860800:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828653.142625:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828653.142641:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828653/real 1787828653]  req@ffff8cd0c8eaf840 x1874671626860800/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828678 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828653.142656:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626860800/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828678 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828653.142665:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828653.142669:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828653.142672:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626860800:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828653.142705:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626860928:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828653.142757:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828653.142770:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828653/real 1787828653]  req@ffff8cd0c8eada40 x1874671626860928/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828678 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828653.142783:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626860928/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828678 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828653.142792:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828653.142796:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828653.142799:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626860928:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828653.142813:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626861056:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828653.142835:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828653.142847:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828653/real 1787828653]  req@ffff8cd0c8eac780 x1874671626861056/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828678 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828653.142859:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626861056/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828678 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828653.142880:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828653.142884:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828653.142887:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626861056:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828653.334204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828653.718230:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828654.358188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828654.742226:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828655.382159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828655.766275:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828656.406159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828656.790295:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828657.430168:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828657.814211:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828658.262172:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828658.262183:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828658.262188:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828658.262199:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2544
00000100:00080000:2.0:1787828658.262213:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828658.262239:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828658.262285:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626861184/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828658.262313:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828658.262316:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828658.262317:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828658.262321:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2544
00000100:00080000:2.0:1787828658.262325:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828658.262329:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828658.262339:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828658.262350:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626861184:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828658.262352:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626861312/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828658.262364:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828658.262367:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828658.262368:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828658.262371:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2544
00000100:00080000:2.0:1787828658.262374:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828658.262376:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828658.262379:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828658.262387:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8b40 x1874671626861440/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828658.262394:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828658.262397:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828658.262404:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828658.262407:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2544
00000100:00080000:2.0:1787828658.262411:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828658.262415:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828658.262419:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787828658.262427:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828658.262431:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626861568/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828658.262472:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828658/real 1787828658]  req@ffff8cd0c3da9680 x1874671626861184/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828683 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828658.262487:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626861184/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828683 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828658.262496:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828658.262499:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828658.262502:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626861184:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828658.262523:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626861312:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828658.262539:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828658.262547:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828658/real 1787828658]  req@ffff8cd0c3da83c0 x1874671626861312/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828683 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828658.262555:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626861312/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828683 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828658.262560:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828658.262569:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828658.262571:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626861312:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828658.262576:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626861440:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828658.262592:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828658.262599:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828658/real 1787828658]  req@ffff8cd0c3da8b40 x1874671626861440/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828683 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828658.262606:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8b40 x1874671626861440/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828683 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828658.262612:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828658.262614:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828658.262615:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626861440:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828658.262619:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626861568:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828658.262639:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828658.262651:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828658/real 1787828658]  req@ffff8cd0c3da8f00 x1874671626861568/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828683 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828658.262663:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626861568/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828683 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828658.262673:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828658.262677:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828658.262680:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626861568:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828658.454233:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828658.838546:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828659.478177:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828659.862252:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828660.502184:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828660.886157:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828661.526163:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828661.910288:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828662.550192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828662.934198:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828663.382199:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828663.382210:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828663.382214:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828663.382224:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2549
00000100:00080000:0.0:1787828663.382233:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828663.382240:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828663.382273:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626861696/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828663.382294:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828663.382297:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828663.382298:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828663.382301:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2549
00000100:00080000:0.0:1787828663.382304:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828663.382306:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828663.382314:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828663.382322:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626861824/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828663.382348:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828663.382351:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828663.382354:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1787828663.382356:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626861696:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828663.382358:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2549
00000100:00080000:0.0:1787828663.382364:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828663.382368:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828663.382372:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828663.382388:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d2c0 x1874671626861952/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828663.382395:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828663.382397:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828663.382399:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828663.382401:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2549
00000100:00080000:0.0:1787828663.382404:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828663.382411:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828663.382414:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828663.382423:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626862080/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828663.382433:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828663.382478:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828663/real 1787828663]  req@ffff8cd0c8e9f840 x1874671626861696/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828688 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828663.382498:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626861696/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828688 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828663.382522:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828663.382531:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828663.382536:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626861696:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828663.382561:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626861824:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828663.382588:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828663.382604:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828663/real 1787828663]  req@ffff8cd0c8e9c000 x1874671626861824/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828688 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828663.382618:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626861824/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828688 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828663.382627:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828663.382631:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828663.382635:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626861824:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828663.382642:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626861952:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828663.382677:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828663.382690:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828663/real 1787828663]  req@ffff8cd0c8e9d2c0 x1874671626861952/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828688 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828663.382704:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d2c0 x1874671626861952/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828688 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828663.382712:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828663.382716:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828663.382720:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626861952:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828663.382727:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626862080:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828663.382748:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828663.382761:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828663/real 1787828663]  req@ffff8cd0c8e9f480 x1874671626862080/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828688 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828663.382774:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626862080/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828688 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828663.382783:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828663.382786:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828663.382790:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626862080:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828663.574181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828663.958228:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828664.598212:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828664.982211:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828665.622202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828666.006298:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828666.646208:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828667.030262:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828667.670162:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828668.054193:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828668.502145:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828668.502170:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828668.502173:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828668.502182:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2554
00000100:00080000:0.0:1787828668.502191:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828668.502197:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828668.502228:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626862208/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828668.502252:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828668.502254:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828668.502256:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828668.502258:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2554
00000100:00080000:0.0:1787828668.502261:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828668.502263:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828668.502271:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828668.502282:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626862336/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828668.502291:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828668.502293:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828668.502294:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828668.502296:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2554
00000100:00080000:0.0:1787828668.502301:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1787828668.502302:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626862208:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828668.502305:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828668.502309:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828668.502320:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d2c0 x1874671626862464/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828668.502334:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828668.502337:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828668.502338:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828668.502341:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2554
00000100:00080000:0.0:1787828668.502345:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828668.502348:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828668.502352:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828668.502360:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626862592/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828668.502381:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828668.502419:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828668/real 1787828668]  req@ffff8cd0c8e9cb40 x1874671626862208/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828693 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828668.502449:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626862208/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828693 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828668.502460:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828668.502465:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828668.502469:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626862208:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828668.502496:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626862336:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828668.502519:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828668.502529:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828668/real 1787828668]  req@ffff8cd0c8e9f480 x1874671626862336/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828693 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828668.502550:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626862336/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828693 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828668.502556:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828668.502558:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828668.502560:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626862336:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828668.502565:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626862464:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828668.502577:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828668.502593:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828668/real 1787828668]  req@ffff8cd0c8e9d2c0 x1874671626862464/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828693 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828668.502624:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d2c0 x1874671626862464/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828693 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828668.502630:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828668.502632:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828668.502635:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626862464:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828668.502638:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626862592:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828668.502653:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828668.502660:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828668/real 1787828668]  req@ffff8cd0c8e9c000 x1874671626862592/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828693 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828668.502668:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626862592/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828693 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828668.502673:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828668.502675:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828668.502677:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626862592:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828668.694206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828669.078254:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828669.718181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828670.102221:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828670.742161:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828671.126258:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828671.766242:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828672.150305:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828672.790165:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828673.174263:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828673.622217:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828673.622227:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828673.622233:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828673.622276:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2559
00000100:00080000:3.0:1787828673.622286:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828673.622295:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828673.622332:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626862720/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828673.622364:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828673.622368:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828673.622370:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828673.622375:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2559
00000100:00080000:3.0:1787828673.622380:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828673.622385:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828673.622395:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828673.622409:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626862848/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828673.622421:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828673.622425:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828673.622427:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828673.622433:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2559
00000100:00080000:3.0:1787828673.622437:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1787828673.622442:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626862720:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787828673.622442:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828673.622451:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828673.622467:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626862976/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828673.622493:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828673.622497:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828673.622499:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828673.622504:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2559
00000100:00080000:3.0:1787828673.622509:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1787828673.622513:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787828673.622518:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828673.622525:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828673.622541:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626863104/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828673.622562:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828673/real 1787828673]  req@ffff8cd0c8eaf0c0 x1874671626862720/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828698 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828673.622577:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626862720/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828698 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828673.622586:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828673.622588:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828673.622591:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626862720:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828673.622597:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626862848:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828673.622613:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828673.622621:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828673/real 1787828673]  req@ffff8cd0c8eac3c0 x1874671626862848/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828698 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828673.622640:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626862848/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828698 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828673.622645:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828673.622648:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828673.622650:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626862848:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828673.622670:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626862976:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828673.622683:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828673.622690:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828673/real 1787828673]  req@ffff8cd0c8eac780 x1874671626862976/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828698 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828673.622697:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626862976/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828698 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828673.622703:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828673.622704:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828673.622706:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626862976:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828673.622710:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626863104:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828673.622719:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828673.622725:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828673/real 1787828673]  req@ffff8cd0c8eada40 x1874671626863104/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828698 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828673.622732:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626863104/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828698 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828673.622747:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828673.622749:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828673.622751:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626863104:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828673.814176:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828674.198212:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828674.838174:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828675.222343:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828675.862221:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828676.246204:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828676.886222:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828677.270210:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828677.910209:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828678.294224:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828678.742153:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828678.742161:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828678.742165:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828678.742173:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2564
00000100:00080000:3.0:1787828678.742182:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828678.742188:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828678.742220:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626863232/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828678.742242:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828678.742245:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828678.742265:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828678.742268:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2564
00000100:00080000:3.0:1787828678.742272:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828678.742275:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828678.742286:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828678.742294:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626863232:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828678.742301:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626863360/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828678.742316:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828678.742320:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828678.742322:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828678.742326:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2564
00000100:00080000:3.0:1787828678.742340:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828678.742349:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:2.0:1787828678.742353:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:3.0:1787828678.742357:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828678.742371:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626863488/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828678.742378:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828678.742381:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828678.742383:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828678.742386:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2564
00000100:00080000:3.0:1787828678.742389:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828678.742392:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828678.742395:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828678/real 1787828678]  req@ffff8cd0c8eaf840 x1874671626863232/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828703 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1787828678.742397:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828678.742413:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626863616/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828678.742423:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626863232/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828703 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828678.742433:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828678.742436:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828678.742440:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626863232:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828678.742461:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626863360:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828678.742476:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828678.742485:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828678/real 1787828678]  req@ffff8cd0c8eae1c0 x1874671626863360/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828703 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828678.742492:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626863360/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828703 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828678.742498:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828678.742506:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828678.742508:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626863360:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828678.742513:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626863488:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828678.742528:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828678.742535:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828678/real 1787828678]  req@ffff8cd0c8ead2c0 x1874671626863488/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828703 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828678.742542:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626863488/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828703 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828678.742547:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828678.742549:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828678.742550:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626863488:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828678.742554:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626863616:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828678.742564:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828678.742570:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828678/real 1787828678]  req@ffff8cd0c8eae940 x1874671626863616/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828703 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828678.742576:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626863616/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828703 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828678.742582:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828678.742583:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828678.742585:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626863616:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828678.934146:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828679.318207:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828679.958197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828680.342207:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828680.982177:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828681.366242:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828682.006197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828682.390224:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828683.030166:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828683.414370:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828683.862160:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828683.862168:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828683.862171:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828683.862179:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2569
00000100:00080000:3.0:1787828683.862188:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828683.862194:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828683.862228:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626863744/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828683.862250:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828683.862253:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828683.862254:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828683.862257:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2569
00000100:00080000:3.0:1787828683.862261:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828683.862266:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828683.862276:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828683.862290:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626863872/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828683.862303:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626863744:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787828683.862304:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828683.862323:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828683.862326:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828683.862330:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2569
00000100:00080000:3.0:1787828683.862335:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828683.862338:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828683.862343:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828683.862354:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626864000/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828683.862362:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828683.862365:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828683.862368:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1787828683.862372:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787828683.862377:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2569
00000100:00080000:3.0:1787828683.862382:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828683.862388:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828683.862393:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828683.862406:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626864128/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828683.862418:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828683/real 1787828683]  req@ffff8cd0c8eac000 x1874671626863744/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828708 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828683.862438:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626863744/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828708 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828683.862450:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828683.862454:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828683.862458:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626863744:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828683.862483:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626863872:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828683.862509:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828683.862523:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828683/real 1787828683]  req@ffff8cd0c8eacf00 x1874671626863872/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828708 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828683.862534:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626863872/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828708 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828683.862543:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828683.862547:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828683.862551:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626863872:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828683.862566:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626864000:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828683.862588:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828683.862610:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828683/real 1787828683]  req@ffff8cd0c8eae940 x1874671626864000/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828708 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828683.862634:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626864000/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828708 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828683.862643:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828683.862646:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828683.862649:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626864000:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828683.862655:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626864128:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828683.862676:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828683.862687:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828683/real 1787828683]  req@ffff8cd0c8ead2c0 x1874671626864128/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828708 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828683.862699:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626864128/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828708 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828683.862707:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828683.862710:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828683.862714:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626864128:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828684.054175:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828684.438191:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828685.078151:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828685.462291:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828686.102167:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828686.486179:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828687.126185:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828687.510203:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828688.150226:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828688.534195:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828688.982136:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828688.982144:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828688.982163:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828688.982171:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2574
00000100:00080000:3.0:1787828688.982180:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828688.982186:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828688.982216:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626864256/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828688.982242:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828688.982246:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828688.982248:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828688.982252:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2574
00000100:00080000:3.0:1787828688.982255:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828688.982259:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828688.982270:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828688.982284:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626864384/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828688.982294:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1787828688.982297:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626864256:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787828688.982297:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828688.982299:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828688.982303:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2574
00000100:00080000:3.0:1787828688.982306:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828688.982310:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828688.982314:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828688.982322:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626864512/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828688.982343:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828688.982345:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828688.982347:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828688.982351:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2574
00000800:00000100:2.0:1787828688.982355:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787828688.982355:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828688.982359:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828688.982363:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828688.982374:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626864640/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828688.982391:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828688/real 1787828688]  req@ffff8cd0c8eae1c0 x1874671626864256/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828713 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828688.982422:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626864256/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828713 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828688.982432:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828688.982437:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828688.982440:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626864256:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828688.982478:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626864384:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828688.982499:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828688.982509:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828688/real 1787828688]  req@ffff8cd0c8eaf840 x1874671626864384/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828713 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828688.982518:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626864384/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828713 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828688.982524:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828688.982526:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828688.982529:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626864384:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828688.982532:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626864512:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828688.982556:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828688.982562:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828688/real 1787828688]  req@ffff8cd0c8eada40 x1874671626864512/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828713 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828688.982570:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626864512/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828713 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828688.982585:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828688.982587:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828688.982590:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626864512:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828688.982594:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626864640:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828688.982614:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828688.982621:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828688/real 1787828688]  req@ffff8cd0c8eac780 x1874671626864640/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828713 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828688.982629:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626864640/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828713 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828688.982634:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828688.982636:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828688.982638:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626864640:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828689.174178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828689.558250:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828690.198169:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828690.582217:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828691.222158:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828691.606268:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828692.246172:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828692.630172:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828693.270221:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828693.654383:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828694.102345:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828694.102360:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828694.102367:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828694.102381:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2579
00000100:00080000:2.0:1787828694.102412:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828694.102422:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828694.102466:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626864768/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828694.102566:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828694.102570:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828694.102572:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828694.102576:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2579
00000100:00080000:2.0:1787828694.102582:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828694.102586:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828694.102597:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828694.102612:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626864896/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828694.102630:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787828694.102630:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626864768:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828694.102645:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828694.102649:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828694.102653:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2579
00000100:00080000:2.0:1787828694.102656:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828694.102659:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828694.102681:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828694.102695:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626865024/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828694.102704:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828694.102707:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828694.102709:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828694.102714:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2579
00000100:00080000:2.0:1787828694.102730:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828694.102735:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828694.102740:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787828694.102750:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828694.102756:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626865152/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828694.102822:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828694/real 1787828694]  req@ffff8cd0c3daa940 x1874671626864768/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828719 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828694.102842:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626864768/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828719 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828694.102854:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828694.102858:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828694.102862:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626864768:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828694.102940:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626864896:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828694.102982:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828694.102993:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828694/real 1787828694]  req@ffff8cd0c3da8780 x1874671626864896/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828719 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828694.103002:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626864896/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828719 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828694.103009:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828694.103012:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828694.103014:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626864896:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828694.103019:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626865024:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828694.103051:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828694.103061:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828694/real 1787828694]  req@ffff8cd0c3da9a40 x1874671626865024/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828719 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828694.103071:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626865024/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828719 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828694.103079:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828694.103082:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828694.103085:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626865024:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828694.103119:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626865152:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828694.103167:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828694.103176:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828694/real 1787828694]  req@ffff8cd0c3dab0c0 x1874671626865152/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828719 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828694.103193:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626865152/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828719 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828694.103200:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828694.103203:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828694.103205:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626865152:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828694.294186:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828694.678220:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828695.318307:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828695.702263:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828696.342177:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828696.726215:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828697.366130:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828697.750303:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828698.390271:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828698.774361:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828699.222300:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828699.222319:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828699.222358:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828699.222377:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2585
00000100:00080000:0.0:1787828699.222397:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828699.222410:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828699.222492:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626865280/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828699.222536:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828699.222541:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828699.222543:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828699.222548:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2585
00000100:00080000:0.0:1787828699.222553:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828699.222558:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828699.222573:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828699.222588:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626865408/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828699.222599:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828699.222603:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828699.222605:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828699.222608:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2585
00000100:00080000:0.0:1787828699.222611:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828699.222614:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828699.222619:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828699.222652:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626865536/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828699.222702:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828699.222707:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828699.222710:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828699.222716:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2585
00000100:00080000:0.0:1787828699.222723:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828699.222729:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828699.222739:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828699.222767:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626865280:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787828699.222777:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626865664/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828699.222887:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828699.222964:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828699/real 1787828699]  req@ffff8cd0c8e9f840 x1874671626865280/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828724 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828699.222994:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626865280/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828724 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828699.223010:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828699.223017:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828699.223024:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626865280:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828699.223042:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626865408:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828699.223186:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828699.223209:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828699/real 1787828699]  req@ffff8cd0c8e9ed00 x1874671626865408/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828724 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828699.223248:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626865408/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828724 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828699.223259:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828699.223264:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828699.223268:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626865408:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828699.223277:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626865536:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828699.223310:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828699.223323:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828699/real 1787828699]  req@ffff8cd0c8e9c3c0 x1874671626865536/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828724 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828699.223338:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626865536/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828724 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828699.223348:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828699.223351:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828699.223355:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626865536:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828699.223391:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626865664:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828699.223413:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828699.223426:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828699/real 1787828699]  req@ffff8cd0c8e9d680 x1874671626865664/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828724 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828699.223443:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626865664/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828724 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828699.223489:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828699.223496:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828699.223502:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626865664:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828699.414144:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828699.798329:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828700.438168:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828700.822317:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828701.462141:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828701.846417:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828702.486141:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828702.870240:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828703.510145:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828703.894267:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828704.342201:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828704.342210:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828704.342213:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828704.342222:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2590
00000100:00080000:0.0:1787828704.342232:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828704.342238:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828704.342282:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626865792/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828704.342305:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828704.342308:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828704.342309:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828704.342312:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2590
00000100:00080000:0.0:1787828704.342329:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828704.342333:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828704.342343:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828704.342353:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626865920/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828704.342364:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828704.342368:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828704.342370:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828704.342378:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2590
00000100:00100000:3.0:1787828704.342378:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626865792:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828704.342382:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828704.342393:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828704.342398:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828704.342410:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626866048/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828704.342418:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828704.342421:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828704.342422:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828704.342426:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2590
00000100:00080000:0.0:1787828704.342428:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828704.342431:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828704.342441:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828704.342449:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626866176/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828704.342471:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828704.342521:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828704/real 1787828704]  req@ffff8cd0c8e9e940 x1874671626865792/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828729 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828704.342536:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626865792/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828729 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828704.342546:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828704.342550:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828704.342553:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626865792:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828704.342577:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626865920:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828704.342607:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828704.342618:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828704/real 1787828704]  req@ffff8cd0c8e9de00 x1874671626865920/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828729 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828704.342627:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626865920/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828729 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828704.342644:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828704.342647:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828704.342649:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626865920:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828704.342654:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626866048:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828704.342672:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828704.342682:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828704/real 1787828704]  req@ffff8cd0c8e9cf00 x1874671626866048/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828729 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828704.342695:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626866048/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828729 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828704.342704:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828704.342707:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828704.342711:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626866048:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828704.342718:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626866176:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828704.342738:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828704.342746:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828704/real 1787828704]  req@ffff8cd0c8e9fc00 x1874671626866176/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828729 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828704.342754:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626866176/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828729 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828704.342760:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828704.342762:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828704.342769:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626866176:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828704.534149:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828704.918200:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828705.558232:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828705.942288:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828706.582218:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828706.966196:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828707.606248:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828707.990302:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828708.630246:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828709.014269:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828709.462216:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828709.462227:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828709.462232:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828709.462242:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2595
00000100:00080000:0.0:1787828709.462254:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828709.462261:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828709.462313:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626866304/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828709.462341:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828709.462345:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828709.462347:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828709.462351:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2595
00000100:00080000:0.0:1787828709.462354:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828709.462357:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828709.462367:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828709.462380:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626866432/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828709.462390:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828709.462394:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828709.462397:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828709.462417:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2595
00000100:00080000:0.0:1787828709.462421:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828709.462426:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828709.462433:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828709.462443:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626866560/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828709.462455:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828709.462458:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828709.462459:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828709.462484:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2595
00000100:00080000:0.0:1787828709.462488:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1787828709.462490:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626866304:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828709.462493:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828709.462505:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828709.462519:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626866688/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828709.462686:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828709.462779:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828709/real 1787828709]  req@ffff8cd0c8e9e580 x1874671626866304/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828734 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828709.462820:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626866304/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828734 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828709.462839:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828709.462846:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828709.462852:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626866304:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828709.462882:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626866432:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828709.462921:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828709.462952:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828709/real 1787828709]  req@ffff8cd0c8e9fc00 x1874671626866432/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828734 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828709.462971:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626866432/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828734 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828709.463001:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828709.463007:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828709.463030:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626866432:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828709.463074:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626866560:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828709.463163:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828709.463181:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828709/real 1787828709]  req@ffff8cd0c8e9cf00 x1874671626866560/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828734 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828709.463212:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626866560/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828734 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828709.463225:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828709.463230:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828709.463236:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626866560:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828709.463245:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626866688:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828709.463277:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828709.463294:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828709/real 1787828709]  req@ffff8cd0c8e9de00 x1874671626866688/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828734 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828709.463313:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626866688/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828734 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828709.463324:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828709.463330:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828709.463335:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626866688:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828709.654227:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828710.038389:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828710.678235:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828711.062203:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828711.702215:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828712.086279:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828712.726222:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828713.110252:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828713.750166:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828714.134235:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828714.582185:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828714.582195:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828714.582199:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828714.582209:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2600
00000100:00080000:2.0:1787828714.582222:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828714.582229:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828714.582271:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626866816/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828714.582295:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828714.582297:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828714.582299:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828714.582301:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2600
00000100:00080000:2.0:1787828714.582303:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828714.582306:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828714.582315:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828714.582324:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626866944/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828714.582357:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828714.582360:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828714.582362:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828714.582365:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2600
00000100:00080000:2.0:1787828714.582368:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828714.582371:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828714.582375:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828714.582383:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626867072/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828714.582396:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828714.582400:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:3.0:1787828714.582400:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626866816:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828714.582403:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828714.582409:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2600
00000100:00080000:2.0:1787828714.582419:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828714.582423:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828714.582426:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828714.582436:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626867200/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828714.582564:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626866944:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828714.582617:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626867072:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828714.582652:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626867200:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828714.774250:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828715.158210:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828715.798180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828716.182276:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828716.822237:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828717.206183:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:0.0:1787828717.654384:0:2345:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:0.0:1787828717.654447:0:2345:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:0.0:1787828717.654469:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:0.0:1787828717.654531:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:0.0:1787828717.654544:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:0.0:1787828717.654577:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:3.0:1787828717.654590:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828714/real 1787828717]  req@ffff8cd0c3dab840 x1874671626867072/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828739 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828717.654622:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626867072/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828739 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828717.654635:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828717.654642:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828717.654647:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626867072:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828717.654657:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828714/real 1787828717]  req@ffff8cd0c3da8000 x1874671626867200/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828739 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828717.654670:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626867200/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828739 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828717.654677:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828717.654691:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828717.654709:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626867200:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828717.654716:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828714/real 1787828717]  req@ffff8cd0c3dabc00 x1874671626866816/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828739 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828717.654733:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626866816/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828739 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828717.654746:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828717.654750:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828717.654755:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626866816:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828717.654761:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828714/real 1787828717]  req@ffff8cd0c3daa580 x1874671626866944/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828739 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828717.654773:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626866944/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828739 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828717.654793:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828717.654797:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828717.654800:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626866944:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828717.846197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828718.230348:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828718.870175:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828719.254205:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828719.702220:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828719.702229:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828719.702233:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828719.702242:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2605
00000100:00080000:0.0:1787828719.702251:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828719.702257:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828719.702287:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626867328/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828719.702311:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828719.702314:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828719.702316:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828719.702318:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2605
00000100:00080000:0.0:1787828719.702321:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828719.702323:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828719.702332:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828719.702341:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626867456/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828719.702351:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828719.702356:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:3.0:1787828719.702356:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626867328:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828719.702360:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828719.702367:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2605
00000100:00080000:0.0:1787828719.702370:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828719.702387:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828719.702392:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828719.702403:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626867584/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828719.702412:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828719.702415:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828719.702418:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828719.702424:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2605
00000800:00000100:3.0:1787828719.702426:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787828719.702433:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828719.702440:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828719.702446:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828719.702460:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626867712/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828719.702499:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828719/real 1787828719]  req@ffff8cd0c8e9e940 x1874671626867328/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828744 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828719.702538:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626867328/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828744 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828719.702550:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828719.702555:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828719.702559:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626867328:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828719.702586:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626867456:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828719.702622:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828719.702638:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828719/real 1787828719]  req@ffff8cd0c8e9d680 x1874671626867456/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828744 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828719.702652:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626867456/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828744 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828719.702661:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828719.702664:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828719.702667:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626867456:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828719.702674:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626867584:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828719.702696:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828719.702708:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828719/real 1787828719]  req@ffff8cd0c8e9c3c0 x1874671626867584/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828744 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828719.702720:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626867584/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828744 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828719.702742:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828719.702746:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828719.702750:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626867584:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828719.702756:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626867712:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828719.702787:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828719.702799:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828719/real 1787828719]  req@ffff8cd0c8e9ed00 x1874671626867712/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828744 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828719.702811:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626867712/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828744 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828719.702819:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828719.702822:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828719.702825:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626867712:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828719.894183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828720.278268:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828720.918194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828721.302176:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828721.942192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828722.326327:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828722.966224:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828723.350185:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828723.990202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828724.374348:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828724.822171:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828724.822181:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828724.822185:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828724.822197:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2610
00000100:00080000:0.0:1787828724.822210:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828724.822217:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828724.822259:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626867840/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828724.822305:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828724.822323:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828724.822325:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828724.822329:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2610
00000100:00080000:0.0:1787828724.822333:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828724.822337:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828724.822348:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828724.822354:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626867840:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787828724.822362:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626867968/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828724.822372:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828724.822375:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828724.822377:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828724.822380:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2610
00000100:00080000:0.0:1787828724.822382:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828724.822384:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828724.822388:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828724.822396:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d2c0 x1874671626868096/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828724.822403:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828724.822406:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828724.822407:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828724.822410:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2610
00000100:00080000:0.0:1787828724.822421:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828724.822424:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:3.0:1787828724.822430:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:0.0:1787828724.822437:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828724.822462:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626868224/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828724.822510:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828724/real 1787828724]  req@ffff8cd0c8e9f840 x1874671626867840/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828749 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828724.822527:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626867840/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828749 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828724.822543:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828724.822547:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828724.822550:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626867840:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828724.822574:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626867968:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828724.822594:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828724.822604:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828724/real 1787828724]  req@ffff8cd0c8e9c000 x1874671626867968/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828749 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828724.822613:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626867968/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828749 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828724.822619:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828724.822621:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828724.822624:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626867968:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828724.822627:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626868096:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828724.822639:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828724.822645:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828724/real 1787828724]  req@ffff8cd0c8e9d2c0 x1874671626868096/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828749 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828724.822653:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d2c0 x1874671626868096/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828749 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828724.822658:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828724.822660:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828724.822662:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626868096:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828724.822672:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626868224:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828724.822686:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828724.822693:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828724/real 1787828724]  req@ffff8cd0c8e9f480 x1874671626868224/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828749 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828724.822702:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626868224/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828749 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828724.822707:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828724.822709:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828724.822710:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626868224:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828725.014227:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828725.398183:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828726.038159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828726.422288:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828727.062164:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828727.446215:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828728.086186:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828728.470309:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828729.110169:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828729.494193:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828729.942203:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828729.942214:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828729.942219:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828729.942229:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2615
00000100:00080000:2.0:1787828729.942266:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828729.942275:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828729.942312:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626868352/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828729.942338:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828729.942342:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828729.942344:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828729.942348:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2615
00000100:00080000:2.0:1787828729.942353:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828729.942358:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828729.942369:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828729.942386:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626868480/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828729.942391:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626868352:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828729.942407:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828729.942412:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828729.942427:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828729.942432:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2615
00000100:00080000:2.0:1787828729.942437:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828729.942443:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828729.942449:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828729.942462:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626868608/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828729.942473:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787828729.942478:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828729.942489:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828729.942492:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828729.942497:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2615
00000100:00080000:2.0:1787828729.942502:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828729.942506:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828729.942512:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828729.942527:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828729/real 1787828729]  req@ffff8cd0c3da9e00 x1874671626868352/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828754 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828729.942528:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daad00 x1874671626868736/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828729.942562:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626868352/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828754 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828729.942575:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828729.942580:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828729.942584:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626868352:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828729.942613:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626868480:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828729.942637:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828729.942649:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828729/real 1787828729]  req@ffff8cd0c3da92c0 x1874671626868480/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828754 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828729.942662:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626868480/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828754 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828729.942671:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828729.942674:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828729.942678:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626868480:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828729.942684:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626868608:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828729.942701:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828729.942710:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828729/real 1787828729]  req@ffff8cd0c3daa1c0 x1874671626868608/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828754 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828729.942722:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626868608/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828754 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828729.942727:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828729.942729:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828729.942732:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626868608:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828729.942735:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626868736:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828729.942748:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828729.942755:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828729/real 1787828729]  req@ffff8cd0c3daad00 x1874671626868736/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828754 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828729.942768:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daad00 x1874671626868736/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828754 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828729.942774:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828729.942776:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828729.942778:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626868736:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828730.134125:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828730.518216:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828731.158180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828731.542251:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828732.182183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828732.566266:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828733.206173:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828733.590227:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828734.230147:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828734.614254:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828735.062153:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828735.062175:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828735.062182:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828735.062197:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2620
00000100:00080000:1.0:1787828735.062212:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828735.062240:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828735.062274:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626868864/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828735.062296:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828735.062309:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828735.062311:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828735.062315:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2620
00000100:00080000:1.0:1787828735.062319:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828735.062323:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828735.062332:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828735.062332:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626868864:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787828735.062349:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626868992/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828735.062362:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828735.062367:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787828735.062369:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828735.062374:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2620
00000100:00080000:1.0:1787828735.062391:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828735.062396:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828735.062402:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787828735.062410:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787828735.062415:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626869120/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828735.062423:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828735.062426:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828735.062428:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828735.062431:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2620
00000100:00080000:1.0:1787828735.062434:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828735.062437:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828735.062446:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828735/real 1787828735]  req@ffff8cd0d48f4b40 x1874671626868864/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828760 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1787828735.062448:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828735.062461:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626869248/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828735.062467:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626868864/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828760 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828735.062481:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828735.062486:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828735.062491:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626868864:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828735.062514:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626868992:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828735.062540:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828735.062552:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828735/real 1787828735]  req@ffff8cd0d48f6580 x1874671626868992/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828760 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828735.062568:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626868992/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828760 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828735.062574:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828735.062577:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828735.062579:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626868992:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828735.062583:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626869120:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828735.062598:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828735.062605:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828735/real 1787828735]  req@ffff8cd0d48f7480 x1874671626869120/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828760 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828735.062612:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626869120/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828760 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828735.062617:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828735.062619:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828735.062621:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626869120:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828735.062625:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626869248:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828735.062644:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828735.062650:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828735/real 1787828735]  req@ffff8cd0d48f4000 x1874671626869248/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828760 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828735.062657:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626869248/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828760 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828735.062670:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828735.062672:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828735.062675:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626869248:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828735.254169:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828735.638203:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828736.278184:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828736.662228:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828737.302203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828737.686243:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828738.326219:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828738.710246:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828739.350226:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828739.734243:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828740.182184:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828740.182192:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828740.182196:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828740.182222:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2626
00000100:00080000:1.0:1787828740.182233:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828740.182241:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828740.182281:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626869376/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828740.182320:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828740.182323:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828740.182325:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828740.182328:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2626
00000100:00080000:1.0:1787828740.182331:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828740.182333:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828740.182358:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828740.182373:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626869504/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828740.182413:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626869376:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787828740.182418:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828740.182426:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787828740.182445:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828740.182452:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2626
00000100:00080000:1.0:1787828740.182458:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828740.182463:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828740.182470:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828740.182499:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626869632/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828740.182509:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:3.0:1787828740.182511:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787828740.182514:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828740.182518:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828740.182522:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2626
00000100:00080000:1.0:1787828740.182526:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828740.182529:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828740.182533:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828740.182559:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626869760/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828740.182584:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828740/real 1787828740]  req@ffff8cd0d48f4000 x1874671626869376/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828765 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828740.182600:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626869376/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828765 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828740.182637:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828740.182641:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828740.182644:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626869376:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828740.182667:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626869504:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828740.182690:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828740.182701:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828740/real 1787828740]  req@ffff8cd0d48f7480 x1874671626869504/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828765 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828740.182711:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626869504/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828765 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828740.182726:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828740.182729:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828740.182732:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626869504:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828740.182738:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626869632:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828740.182756:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828740.182765:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828740/real 1787828740]  req@ffff8cd0d48f6580 x1874671626869632/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828765 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828740.182774:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626869632/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828765 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828740.182781:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828740.182783:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828740.182785:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626869632:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828740.182790:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626869760:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828740.182803:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828740.182811:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828740/real 1787828740]  req@ffff8cd0d48f4b40 x1874671626869760/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828765 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828740.182820:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626869760/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828765 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828740.182826:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828740.182828:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828740.182841:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626869760:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828740.374230:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828740.758253:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828741.398172:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828741.782200:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828742.422300:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828742.806322:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828743.446234:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828743.830484:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828744.470219:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828744.854395:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828745.302358:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828745.302370:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828745.302375:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828745.302387:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2631
00000100:00080000:2.0:1787828745.302400:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828745.302419:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828745.302467:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daad00 x1874671626869888/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828745.302515:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828745.302520:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828745.302522:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828745.302536:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2631
00000100:00080000:2.0:1787828745.302540:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828745.302545:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828745.302558:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828745.302573:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626870016/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828745.302608:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828745.302612:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828745.302616:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828745.302626:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2631
00000100:00100000:3.0:1787828745.302628:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626869888:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828745.302632:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828745.302646:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828745.302660:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828745.302677:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626870144/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828745.302689:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828745.302693:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828745.302695:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828745.302700:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2631
00000100:00080000:2.0:1787828745.302717:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828745.302726:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828745.302735:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828745.302758:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626870272/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828745.302762:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828745.302856:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828745/real 1787828745]  req@ffff8cd0c3daad00 x1874671626869888/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828770 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828745.302881:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daad00 x1874671626869888/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828770 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828745.302906:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828745.302911:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828745.302916:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626869888:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828745.302955:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626870016:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828745.302994:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828745.303034:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828745/real 1787828745]  req@ffff8cd0c3daa1c0 x1874671626870016/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828770 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828745.303060:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626870016/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828770 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828745.303078:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828745.303084:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828745.303200:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626870016:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828745.303211:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626870144:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828745.303259:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828745.303273:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828745/real 1787828745]  req@ffff8cd0c3da92c0 x1874671626870144/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828770 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828745.303285:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626870144/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828770 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828745.303293:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828745.303296:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828745.303300:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626870144:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828745.303306:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626870272:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828745.303349:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828745.303370:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828745/real 1787828745]  req@ffff8cd0c3da9e00 x1874671626870272/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828770 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828745.303382:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626870272/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828770 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828745.303389:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828745.303393:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828745.303395:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626870272:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828745.494178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828745.878337:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828746.518266:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828746.902439:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828747.542177:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828747.926247:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828748.566197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828748.950388:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828749.590395:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828749.974260:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828750.422204:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828750.422215:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828750.422219:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828750.422229:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2636
00000100:00080000:2.0:1787828750.422259:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828750.422267:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828750.422316:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626870400/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828750.422345:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828750.422349:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828750.422351:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828750.422356:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2636
00000100:00080000:2.0:1787828750.422360:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828750.422366:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828750.422377:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828750.422399:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626870528/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828750.422399:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626870400:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828750.422427:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828750.422432:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828750.422434:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828750.422440:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2636
00000100:00080000:2.0:1787828750.422444:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828750.422449:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828750.422456:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828750.422470:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626870656/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828750.422481:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:3.0:1787828750.422496:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787828750.422497:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828750.422504:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828750.422511:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2636
00000100:00080000:2.0:1787828750.422519:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828750.422526:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828750.422533:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828750.422552:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626870784/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828750.422559:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828750/real 1787828750]  req@ffff8cd0c3daa580 x1874671626870400/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828775 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828750.422598:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626870400/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828775 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828750.422609:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828750.422612:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828750.422616:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626870400:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828750.422655:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626870528:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828750.422681:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828750.422695:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828750/real 1787828750]  req@ffff8cd0c3dabc00 x1874671626870528/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828775 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828750.422714:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626870528/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828775 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828750.422720:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828750.422723:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828750.422725:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626870528:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828750.422729:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626870656:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828750.422758:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828750.422766:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828750/real 1787828750]  req@ffff8cd0c3da8000 x1874671626870656/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828775 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828750.422774:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626870656/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828775 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828750.422779:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828750.422782:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828750.422784:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626870656:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828750.422788:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626870784:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828750.422799:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828750.422805:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828750/real 1787828750]  req@ffff8cd0c3dab840 x1874671626870784/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828775 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828750.422812:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626870784/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828775 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828750.422817:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828750.422819:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828750.422821:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626870784:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828750.614173:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828750.998354:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828751.638181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828752.022308:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828752.662204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828753.046357:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828753.686324:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828754.070313:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828754.710265:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828755.094319:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828755.542308:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828755.542319:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828755.542324:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828755.542335:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2641
00000100:00080000:2.0:1787828755.542348:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828755.542355:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828755.542396:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626870912/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828755.542423:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828755.542427:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828755.542429:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828755.542432:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2641
00000100:00080000:2.0:1787828755.542436:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828755.542441:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828755.542450:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828755.542465:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626871040/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828755.542473:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626870912:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828755.542498:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828755.542502:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828755.542505:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828755.542509:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2641
00000100:00080000:2.0:1787828755.542513:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828755.542519:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828755.542525:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828755.542537:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626871168/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828755.542547:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828755.542550:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828755.542553:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828755.542557:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2641
00000100:00080000:2.0:1787828755.542571:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828755.542575:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828755.542580:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787828755.542586:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828755.542591:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626871296/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828755.542647:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828755/real 1787828755]  req@ffff8cd0c3dab0c0 x1874671626870912/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828780 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828755.542665:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626870912/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828780 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828755.542676:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828755.542680:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828755.542684:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626870912:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828755.542713:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626871040:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828755.542734:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828755.542744:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828755/real 1787828755]  req@ffff8cd0c3da9a40 x1874671626871040/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828780 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828755.542753:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626871040/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828780 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828755.542759:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828755.542762:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828755.542764:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626871040:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828755.542769:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626871168:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828755.542790:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828755.542798:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828755/real 1787828755]  req@ffff8cd0c3da8780 x1874671626871168/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828780 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828755.542817:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626871168/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828780 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828755.542823:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828755.542825:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828755.542828:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626871168:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828755.542832:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626871296:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828755.542848:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828755.542855:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828755/real 1787828755]  req@ffff8cd0c3daa940 x1874671626871296/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828780 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828755.542863:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626871296/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828780 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828755.542868:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828755.542870:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828755.542872:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626871296:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828755.734217:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828756.118246:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828756.758215:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828757.142287:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828757.782177:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828758.166188:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828758.806211:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828759.190263:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828759.830255:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828760.214288:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828760.662305:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828760.662316:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828760.662321:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828760.662333:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2646
00000100:00080000:1.0:1787828760.662346:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828760.662354:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828760.662404:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626871424/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828760.662434:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828760.662437:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828760.662438:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828760.662441:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2646
00000100:00080000:1.0:1787828760.662443:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828760.662446:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828760.662456:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828760.662466:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626871552/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828760.662474:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828760.662477:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787828760.662479:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828760.662482:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2646
00000100:00080000:1.0:1787828760.662484:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828760.662486:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828760.662489:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828760.662526:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626871424:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787828760.662540:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626871680/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828760.662549:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828760.662552:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828760.662554:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828760.662560:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2646
00000100:00080000:1.0:1787828760.662565:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828760.662570:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828760.662576:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828760.662591:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626871808/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828760.662622:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828760.662676:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828760/real 1787828760]  req@ffff8cd0d48f7840 x1874671626871424/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828785 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828760.662701:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626871424/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828785 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828760.662710:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828760.662714:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828760.662717:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626871424:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828760.662724:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626871552:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828760.662744:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828760.662753:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828760/real 1787828760]  req@ffff8cd0d48f5a40 x1874671626871552/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828785 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828760.662762:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626871552/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828785 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828760.662768:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828760.662770:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828760.662772:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626871552:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828760.662801:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626871680:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828760.662814:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828760.662821:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828760/real 1787828760]  req@ffff8cd0d48f43c0 x1874671626871680/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828785 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828760.662829:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626871680/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828785 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828760.662844:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828760.662847:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828760.662849:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626871680:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828760.662854:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626871808:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828760.662881:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828760.662889:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828760/real 1787828760]  req@ffff8cd0d48f70c0 x1874671626871808/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828785 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828760.662898:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626871808/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828785 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828760.662903:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828760.662906:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828760.662908:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626871808:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828760.854236:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828761.238396:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828761.878235:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828762.262447:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828762.902302:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828763.286320:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828763.926215:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828764.310254:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828764.950155:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828765.334346:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828765.782281:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828765.782290:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828765.782294:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828765.782303:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2651
00000100:00080000:0.0:1787828765.782314:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828765.782321:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828765.782372:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626871936/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828765.782427:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828765.782430:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828765.782432:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828765.782435:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2651
00000100:00080000:0.0:1787828765.782438:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828765.782441:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828765.782450:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828765.782464:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626872064/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828765.782476:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828765.782480:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828765.782483:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828765.782503:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2651
00000100:00080000:0.0:1787828765.782510:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1787828765.782514:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626871936:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828765.782519:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828765.782529:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828765.782546:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d2c0 x1874671626872192/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828765.782557:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828765.782560:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828765.782562:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828765.782572:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2651
00000100:00080000:0.0:1787828765.782577:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828765.782580:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828765.782586:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787828765.782587:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787828765.782602:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626872320/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828765.782632:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828765/real 1787828765]  req@ffff8cd0c8e9cb40 x1874671626871936/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828790 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828765.782649:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626871936/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828790 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828765.782667:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828765.782671:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828765.782674:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626871936:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828765.782681:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626872064:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828765.782702:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828765.782712:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828765/real 1787828765]  req@ffff8cd0c8e9f480 x1874671626872064/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828790 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828765.782733:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626872064/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828790 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828765.782740:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828765.782743:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828765.782745:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626872064:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828765.782766:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626872192:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828765.782780:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828765.782788:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828765/real 1787828765]  req@ffff8cd0c8e9d2c0 x1874671626872192/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828790 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828765.782797:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d2c0 x1874671626872192/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828790 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828765.782804:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828765.782806:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828765.782808:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626872192:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828765.782819:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626872320:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828765.782850:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828765.782857:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828765/real 1787828765]  req@ffff8cd0c8e9c000 x1874671626872320/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828790 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828765.782865:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626872320/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828790 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828765.782871:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828765.782872:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828765.782874:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626872320:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828765.974139:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828766.358277:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828766.998132:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828767.382260:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828768.022148:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828768.406294:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828769.046140:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828769.430244:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828770.070170:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828770.454217:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828770.902244:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828770.902257:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828770.902262:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828770.902275:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2656
00000100:00080000:3.0:1787828770.902290:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828770.902299:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828770.902348:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626872448/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828770.902381:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828770.902385:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828770.902405:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828770.902423:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2656
00000100:00080000:3.0:1787828770.902430:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828770.902442:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828770.902443:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626872448:192.168.203.145@tcp:250:kworker.0
00000008:00100000:3.0:1787828770.902459:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828770.902477:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626872576/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828770.902488:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828770.902491:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828770.902494:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828770.902497:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2656
00000100:00080000:3.0:1787828770.902502:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828770.902506:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:2.0:1787828770.902513:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:3.0:1787828770.902514:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828770.902531:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626872704/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828770.902540:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828770.902543:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828770.902545:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828770.902548:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2656
00000100:00080000:3.0:1787828770.902552:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828770.902556:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828770.902558:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828770/real 1787828770]  req@ffff8cd0c8eac3c0 x1874671626872448/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828795 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:3.0:1787828770.902561:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828770.902581:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626872832/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828770.902641:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626872448/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828795 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828770.902652:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828770.902657:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828770.902660:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626872448:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828770.902682:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626872576:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828770.902701:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828770.902709:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828770/real 1787828770]  req@ffff8cd0c8eaf0c0 x1874671626872576/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828795 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828770.902718:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626872576/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828795 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828770.902723:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828770.902726:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828770.902728:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626872576:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828770.902731:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626872704:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828770.902743:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828770.902749:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828770/real 1787828770]  req@ffff8cd0c8eac780 x1874671626872704/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828795 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828770.902756:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626872704/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828795 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828770.902770:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828770.902773:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828770.902775:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626872704:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828770.902779:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626872832:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828770.902793:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828770.902800:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828770/real 1787828770]  req@ffff8cd0c8eada40 x1874671626872832/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828795 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828770.902807:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626872832/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828795 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828770.902812:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828770.902814:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828770.902816:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626872832:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828771.094203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828771.478271:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828772.118237:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828772.502265:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828773.142226:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828773.526253:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828774.166165:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828774.550277:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828775.190198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828775.574239:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828776.022183:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828776.022191:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828776.022195:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828776.022203:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2661
00000100:00080000:2.0:1787828776.022212:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828776.022218:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828776.022250:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626872960/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828776.022273:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828776.022287:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828776.022289:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828776.022292:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2661
00000100:00080000:2.0:1787828776.022295:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828776.022299:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828776.022307:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626872960:192.168.203.145@tcp:250:kworker.0
00000008:00100000:2.0:1787828776.022308:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828776.022323:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8b40 x1874671626873088/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828776.022332:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828776.022335:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828776.022336:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828776.022339:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2661
00000100:00080000:2.0:1787828776.022342:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828776.022344:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828776.022347:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828776.022355:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626873216/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828776.022361:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828776.022364:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828776.022368:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:3.0:1787828776.022368:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787828776.022372:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2661
00000100:00080000:2.0:1787828776.022377:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828776.022379:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828776.022388:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828776.022397:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626873344/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828776.022418:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828776/real 1787828776]  req@ffff8cd0c3da8f00 x1874671626872960/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828801 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828776.022433:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626872960/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828801 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828776.022442:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828776.022444:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828776.022447:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626872960:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828776.022469:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626873088:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828776.022486:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828776.022495:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828776/real 1787828776]  req@ffff8cd0c3da8b40 x1874671626873088/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828801 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828776.022503:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8b40 x1874671626873088/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828801 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828776.022509:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828776.022517:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828776.022520:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626873088:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828776.022524:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626873216:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828776.022539:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828776.022546:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828776/real 1787828776]  req@ffff8cd0c3da83c0 x1874671626873216/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828801 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828776.022554:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626873216/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828801 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828776.022560:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828776.022562:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828776.022564:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626873216:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828776.022568:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626873344:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828776.022578:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828776.022583:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828776/real 1787828776]  req@ffff8cd0c3da9680 x1874671626873344/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828801 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828776.022590:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626873344/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828801 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828776.022595:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828776.022597:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828776.022599:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626873344:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828776.214189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828776.598274:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828777.238189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828777.622238:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828778.262216:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828778.646375:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828779.286162:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828779.670274:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828780.310221:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828780.694225:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828781.142243:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828781.142251:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828781.142262:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828781.142270:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2666
00000100:00080000:3.0:1787828781.142280:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828781.142286:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828781.142320:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626873472/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828781.142346:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828781.142349:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828781.142350:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828781.142353:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2666
00000100:00080000:3.0:1787828781.142356:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828781.142359:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828781.142367:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828781.142375:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626873600/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828781.142385:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828781.142388:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828781.142390:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828781.142396:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2666
00000100:00100000:2.0:1787828781.142399:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626873472:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787828781.142413:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828781.142417:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828781.142423:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828781.142435:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626873728/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828781.142442:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828781.142445:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828781.142446:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828781.142449:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2666
00000100:00080000:3.0:1787828781.142469:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828781.142472:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828781.142476:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828781.142483:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626873856/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828781.142545:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626873600:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828781.142585:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626873728:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828781.142612:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626873856:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828781.334191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828781.718233:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828782.358187:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828782.742265:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828783.382188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828783.766211:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:1.0:1787828784.214400:0:2344:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:1.0:1787828784.214446:0:2344:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1787828784.214480:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787828784.214544:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787828784.214554:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787828784.214571:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:2.0:1787828784.214599:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828781/real 1787828784]  req@ffff8cd0c8eae1c0 x1874671626873600/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828806 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828784.214625:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626873600/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828806 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828784.214637:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828784.214642:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828784.214647:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626873600:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828784.214655:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828781/real 1787828784]  req@ffff8cd0c8ead2c0 x1874671626873728/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828806 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828784.214669:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626873728/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828806 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828784.214677:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828784.214681:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828784.214699:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626873728:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828784.214708:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828781/real 1787828784]  req@ffff8cd0c8eae940 x1874671626873856/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828806 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828784.214731:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626873856/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828806 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828784.214740:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828784.214744:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828784.214748:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626873856:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828784.214756:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828781/real 1787828784]  req@ffff8cd0c8eaf840 x1874671626873472/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828806 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828784.214769:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626873472/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828806 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828784.214778:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828784.214782:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828784.214785:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626873472:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828784.406169:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828784.790297:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828785.430161:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828785.814277:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828786.262202:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828786.262214:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828786.262218:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828786.262229:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2672
00000100:00080000:2.0:1787828786.262238:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828786.262244:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828786.262275:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626873984/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828786.262311:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828786.262315:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828786.262316:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828786.262319:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2672
00000100:00080000:2.0:1787828786.262323:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828786.262336:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828786.262345:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828786.262357:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626873984:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828786.262359:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626874112/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828786.262373:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828786.262375:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828786.262376:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828786.262379:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2672
00000100:00080000:2.0:1787828786.262383:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828786.262387:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828786.262392:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828786.262405:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8b40 x1874671626874240/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828786.262416:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:3.0:1787828786.262419:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787828786.262424:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828786.262428:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828786.262434:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2672
00000100:00080000:2.0:1787828786.262448:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828786.262455:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828786.262459:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828786/real 1787828786]  req@ffff8cd0c3da9680 x1874671626873984/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828811 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1787828786.262463:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828786.262480:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626873984/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828811 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828786.262486:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626874368/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828786.262510:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828786.262516:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828786.262521:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626873984:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828786.262540:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626874112:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828786.262562:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828786.262576:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828786/real 1787828786]  req@ffff8cd0c3da83c0 x1874671626874112/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828811 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828786.262589:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626874112/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828811 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828786.262597:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828786.262601:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828786.262604:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626874112:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828786.262610:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626874240:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828786.262632:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828786.262645:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828786/real 1787828786]  req@ffff8cd0c3da8b40 x1874671626874240/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828811 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828786.262657:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8b40 x1874671626874240/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828811 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828786.262665:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828786.262669:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828786.262672:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626874240:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828786.262685:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626874368:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828786.262709:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828786.262721:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828786/real 1787828786]  req@ffff8cd0c3da8f00 x1874671626874368/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828811 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828786.262733:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626874368/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828811 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828786.262741:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828786.262744:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828786.262748:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626874368:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828786.454229:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828786.838356:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828787.478197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828787.862244:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828788.502234:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828788.886244:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828789.526206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828789.910228:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828790.550213:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828790.934303:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828791.382246:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828791.382260:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828791.382265:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828791.382277:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2677
00000100:00080000:3.0:1787828791.382290:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828791.382297:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828791.382342:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626874496/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828791.382376:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828791.382378:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828791.382400:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828791.382404:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2677
00000100:00080000:3.0:1787828791.382407:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828791.382410:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828791.382421:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828791.382433:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626874496:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828791.382435:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626874624/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828791.382458:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828791.382461:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828791.382463:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828791.382467:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2677
00000100:00080000:3.0:1787828791.382470:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828791.382473:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828791.382478:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828791.382490:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626874752/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828791.382497:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828791.382500:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828791.382502:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828791.382505:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2677
00000100:00080000:3.0:1787828791.382507:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828791.382514:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:2.0:1787828791.382516:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:3.0:1787828791.382520:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828791.382540:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626874880/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828791.382573:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828791/real 1787828791]  req@ffff8cd0c8eacf00 x1874671626874496/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828816 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828791.382595:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626874496/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828816 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828791.382604:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828791.382608:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828791.382611:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626874496:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828791.382633:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626874624:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828791.382653:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828791.382663:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828791/real 1787828791]  req@ffff8cd0c8eac000 x1874671626874624/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828816 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828791.382672:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626874624/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828816 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828791.382678:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828791.382680:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828791.382695:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626874624:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828791.382700:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626874752:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828791.382725:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828791.382733:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828791/real 1787828791]  req@ffff8cd0c8eaf840 x1874671626874752/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828816 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828791.382742:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626874752/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828816 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828791.382748:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828791.382750:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828791.382752:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626874752:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828791.382756:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626874880:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828791.382767:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828791.382773:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828791/real 1787828791]  req@ffff8cd0c8eae940 x1874671626874880/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828816 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828791.382781:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626874880/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828816 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828791.382786:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828791.382788:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828791.382790:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626874880:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828791.574235:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828791.958187:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828792.598172:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828792.982239:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828793.622155:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828794.006217:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828794.646185:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828795.030238:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828795.670191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828796.054237:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828796.502192:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828796.502201:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828796.502205:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828796.502213:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2682
00000100:00080000:3.0:1787828796.502222:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828796.502229:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828796.502262:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626875008/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828796.502286:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828796.502288:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828796.502290:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828796.502292:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2682
00000100:00080000:3.0:1787828796.502295:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828796.502297:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828796.502305:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828796.502314:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626875136/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828796.502323:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1787828796.502328:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626875008:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787828796.502339:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828796.502341:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828796.502345:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2682
00000100:00080000:3.0:1787828796.502349:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828796.502351:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828796.502356:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828796.502367:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626875264/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828796.502375:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828796.502378:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828796.502379:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828796.502381:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2682
00000100:00080000:3.0:1787828796.502384:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1787828796.502387:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787828796.502390:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828796.502393:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828796.502404:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626875392/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828796.502441:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828796/real 1787828796]  req@ffff8cd0c8ead2c0 x1874671626875008/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828821 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828796.502462:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626875008/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828821 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828796.502472:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828796.502484:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828796.502487:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626875008:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828796.502507:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626875136:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828796.502527:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828796.502537:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828796/real 1787828796]  req@ffff8cd0c8eae1c0 x1874671626875136/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828821 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828796.502546:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626875136/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828821 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828796.502552:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828796.502554:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828796.502556:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626875136:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828796.502561:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626875264:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828796.502582:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828796.502595:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828796/real 1787828796]  req@ffff8cd0c8eada40 x1874671626875264/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828821 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828796.502609:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626875264/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828821 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828796.502634:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828796.502638:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828796.502640:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626875264:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828796.502645:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626875392:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828796.502662:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828796.502676:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828796/real 1787828796]  req@ffff8cd0c8eac780 x1874671626875392/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828821 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828796.502684:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626875392/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828821 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828796.502690:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828796.502692:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828796.502694:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626875392:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828796.694179:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828797.078235:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828797.718219:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828798.102206:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828798.742204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828799.126202:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828799.766250:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828800.150318:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828800.790237:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828801.174239:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828801.622201:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828801.622211:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828801.622216:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828801.622226:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2687
00000100:00080000:2.0:1787828801.622239:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828801.622247:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828801.622302:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626875520/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828801.622337:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828801.622355:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828801.622357:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828801.622361:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2687
00000100:00080000:2.0:1787828801.622366:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828801.622370:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828801.622378:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828801.622381:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626875520:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828801.622407:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626875648/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828801.622417:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828801.622419:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828801.622420:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828801.622423:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2687
00000100:00080000:2.0:1787828801.622426:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828801.622429:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828801.622433:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828801.622441:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626875776/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828801.622451:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828801.622455:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828801.622457:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828801.622461:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2687
00000800:00000100:3.0:1787828801.622467:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787828801.622468:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828801.622492:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828801.622507:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828801.622523:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626875904/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828801.622534:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828801/real 1787828801]  req@ffff8cd0c3daa940 x1874671626875520/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828826 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828801.622552:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626875520/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828826 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828801.622561:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828801.622564:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828801.622567:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626875520:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828801.622586:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626875648:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828801.622626:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828801.622640:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828801/real 1787828801]  req@ffff8cd0c3da8780 x1874671626875648/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828826 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828801.622653:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626875648/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828826 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828801.622662:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828801.622665:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828801.622669:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626875648:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828801.622676:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626875776:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828801.622698:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828801.622710:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828801/real 1787828801]  req@ffff8cd0c3da9a40 x1874671626875776/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828826 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828801.622718:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626875776/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828826 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828801.622735:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828801.622738:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828801.622740:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626875776:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828801.622744:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626875904:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828801.622760:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828801.622767:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828801/real 1787828801]  req@ffff8cd0c3dab0c0 x1874671626875904/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828826 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828801.622775:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626875904/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828826 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828801.622780:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828801.622782:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828801.622783:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626875904:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828801.814161:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828802.198310:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828802.838208:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828803.222271:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828803.862218:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828804.246374:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828804.886303:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828805.270310:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828805.910204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828806.294337:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828806.742242:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828806.742254:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828806.742259:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828806.742271:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2692
00000100:00080000:0.0:1787828806.742285:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828806.742292:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828806.742339:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626876032/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828806.742371:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828806.742375:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828806.742376:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828806.742379:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2692
00000100:00080000:0.0:1787828806.742382:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828806.742385:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828806.742397:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828806.742422:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626876160/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828806.742438:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626876032:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828806.742439:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828806.742464:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828806.742466:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828806.742471:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2692
00000100:00080000:0.0:1787828806.742475:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828806.742479:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828806.742485:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828806.742505:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626876288/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828806.742514:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828806.742517:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828806.742519:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828806.742525:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2692
00000800:00000100:3.0:1787828806.742527:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787828806.742529:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828806.742535:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828806.742539:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828806.742548:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626876416/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828806.742592:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828806/real 1787828806]  req@ffff8cd0c8e9f840 x1874671626876032/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828831 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828806.742610:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626876032/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828831 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828806.742641:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828806.742645:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828806.742649:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626876032:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828806.742679:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626876160:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828806.742703:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828806.742714:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828806/real 1787828806]  req@ffff8cd0c8e9ed00 x1874671626876160/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828831 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828806.742724:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626876160/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828831 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828806.742731:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828806.742741:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828806.742744:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626876160:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828806.742750:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626876288:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828806.742766:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828806.742775:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828806/real 1787828806]  req@ffff8cd0c8e9c3c0 x1874671626876288/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828831 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828806.742783:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626876288/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828831 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828806.742790:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828806.742792:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828806.742794:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626876288:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828806.742806:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626876416:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828806.742822:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828806.742830:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828806/real 1787828806]  req@ffff8cd0c8e9d680 x1874671626876416/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828831 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828806.742838:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626876416/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828831 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828806.742844:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828806.742846:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828806.742848:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626876416:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828806.934146:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828807.318342:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828807.958133:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828808.342267:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828808.982228:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828809.366289:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828810.006239:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828810.390308:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828811.030209:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828811.414394:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828811.862204:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828811.862219:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828811.862225:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828811.862239:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2697
00000100:00080000:0.0:1787828811.862253:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828811.862261:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828811.862310:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626876544/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828811.862380:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828811.862384:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828811.862387:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828811.862392:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2697
00000100:00080000:0.0:1787828811.862397:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828811.862403:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828811.862414:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828811.862428:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626876672/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828811.862437:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626876544:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828811.862440:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828811.862445:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828811.862446:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828811.862450:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2697
00000100:00080000:0.0:1787828811.862454:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828811.862458:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828811.862464:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828811.862479:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626876800/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828811.862490:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828811.862495:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828811.862498:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828811.862502:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2697
00000100:00080000:0.0:1787828811.862512:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828811.862516:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828811.862521:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787828811.862524:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787828811.862535:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626876928/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828811.862574:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828811/real 1787828811]  req@ffff8cd0c8e9e940 x1874671626876544/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828836 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828811.862591:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626876544/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828836 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828811.862601:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828811.862605:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828811.862608:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626876544:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828811.862640:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626876672:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828811.862660:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828811.862678:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828811/real 1787828811]  req@ffff8cd0c8e9de00 x1874671626876672/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828836 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828811.862687:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626876672/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828836 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828811.862692:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828811.862694:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828811.862696:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626876672:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828811.862700:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626876800:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828811.862713:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828811.862719:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828811/real 1787828811]  req@ffff8cd0c8e9cf00 x1874671626876800/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828836 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828811.862727:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626876800/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828836 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828811.862732:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828811.862734:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828811.862736:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626876800:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828811.862740:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626876928:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828811.862750:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828811.862756:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828811/real 1787828811]  req@ffff8cd0c8e9fc00 x1874671626876928/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828836 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828811.862780:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626876928/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828836 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828811.862785:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828811.862788:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828811.862790:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626876928:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828812.054206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828812.438296:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828813.078152:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828813.462267:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828814.102219:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828814.486220:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828815.126301:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828815.510308:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828816.150229:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828816.534360:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828816.982211:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828816.982222:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828816.982227:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828816.982237:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2702
00000100:00080000:2.0:1787828816.982248:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828816.982255:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828816.982298:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626877056/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828816.982326:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828816.982329:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828816.982330:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828816.982333:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2702
00000100:00080000:2.0:1787828816.982336:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828816.982339:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828816.982349:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828816.982361:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626877184/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828816.982364:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626877056:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828816.982389:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828816.982393:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828816.982394:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828816.982399:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2702
00000100:00080000:2.0:1787828816.982428:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828816.982433:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828816.982439:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787828816.982442:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828816.982452:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626877312/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828816.982461:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828816.982464:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828816.982466:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828816.982469:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2702
00000100:00080000:2.0:1787828816.982472:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828816.982475:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828816.982479:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828816.982492:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626877440/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828816.982502:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828816/real 1787828816]  req@ffff8cd0c3dab840 x1874671626877056/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828841 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828816.982521:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626877056/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828841 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828816.982530:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828816.982533:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828816.982536:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626877056:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828816.982558:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626877184:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828816.982579:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828816.982588:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828816/real 1787828816]  req@ffff8cd0c3da8000 x1874671626877184/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828841 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828816.982609:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626877184/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828841 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828816.982615:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828816.982618:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828816.982620:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626877184:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828816.982624:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626877312:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828816.982639:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828816.982646:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828816/real 1787828816]  req@ffff8cd0c3dabc00 x1874671626877312/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828841 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828816.982654:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626877312/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828841 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828816.982659:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828816.982661:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828816.982663:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626877312:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828816.982667:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626877440:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828816.982677:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828816.982683:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828816/real 1787828816]  req@ffff8cd0c3daa580 x1874671626877440/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828841 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828816.982691:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626877440/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828841 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828816.982707:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828816.982709:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828816.982712:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626877440:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828817.174185:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828817.558270:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828818.198221:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828818.582218:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828819.222208:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828819.606320:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828820.246242:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828820.630265:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828821.270164:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828821.654240:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828822.102213:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828822.102228:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828822.102232:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828822.102240:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2707
00000100:00080000:3.0:1787828822.102250:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828822.102256:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828822.102290:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626877568/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828822.102313:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828822.102315:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828822.102317:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828822.102319:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2707
00000100:00080000:3.0:1787828822.102322:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828822.102335:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828822.102343:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828822.102351:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626877696/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828822.102366:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626877568:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787828822.102387:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828822.102400:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828822.102402:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828822.102406:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2707
00000100:00080000:3.0:1787828822.102411:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828822.102415:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828822.102422:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828822.102437:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626877824/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787828822.102447:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787828822.102463:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828822.102468:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828822.102470:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828822.102476:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2707
00000100:00080000:3.0:1787828822.102481:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828822.102485:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828822.102491:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828822.102499:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828822/real 1787828822]  req@ffff8cd0c8eaf0c0 x1874671626877568/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828847 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828822.102506:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626877952/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828822.102529:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626877568/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828847 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828822.102541:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828822.102545:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828822.102550:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626877568:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828822.102578:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626877696:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828822.102600:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828822.102621:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828822/real 1787828822]  req@ffff8cd0c8eac3c0 x1874671626877696/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828847 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828822.102632:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626877696/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828847 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828822.102639:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828822.102642:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828822.102645:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626877696:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828822.102650:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626877824:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828822.102665:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828822.102681:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828822/real 1787828822]  req@ffff8cd0c8eac780 x1874671626877824/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828847 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828822.102691:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626877824/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828847 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828822.102698:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828822.102700:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828822.102703:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626877824:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828822.102707:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626877952:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828822.102720:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828822.102729:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828822/real 1787828822]  req@ffff8cd0c8eada40 x1874671626877952/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828847 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828822.102749:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626877952/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828847 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828822.102758:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828822.102761:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828822.102765:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626877952:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828822.294166:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828822.678251:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828823.318147:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828823.702309:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828824.342163:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828824.726304:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828825.366151:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828825.750236:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828826.390223:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828826.774254:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828827.222207:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828827.222232:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828827.222236:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828827.222244:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2713
00000100:00080000:2.0:1787828827.222254:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828827.222260:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828827.222312:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626878080/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828827.222338:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828827.222341:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828827.222342:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828827.222345:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2713
00000100:00080000:2.0:1787828827.222366:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828827.222371:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828827.222384:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828827.222398:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626878080:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828827.222406:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626878208/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828827.222480:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828827.222485:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828827.222488:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:3.0:1787828827.222492:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787828827.222499:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2713
00000100:00080000:2.0:1787828827.222523:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828827.222528:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828827.222535:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828827.222551:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626878336/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828827.222560:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828827/real 1787828827]  req@ffff8cd0c3da9e00 x1874671626878080/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828852 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828827.222564:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828827.222585:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828827.222587:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828827.222597:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2713
00000100:00080000:2.0:1787828827.222602:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828827.222606:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828827.222611:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626878080/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828852 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1787828827.222613:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00080000:3.0:1787828827.222628:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:2.0:1787828827.222634:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daad00 x1874671626878464/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828827.222636:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828827.222646:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626878080:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828827.222675:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626878208:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828827.222699:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828827.222710:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828827/real 1787828827]  req@ffff8cd0c3da92c0 x1874671626878208/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828852 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828827.222721:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626878208/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828852 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828827.222728:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828827.222732:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828827.222734:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626878208:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828827.222757:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626878336:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828827.222793:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828827.222811:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828827/real 1787828827]  req@ffff8cd0c3daa1c0 x1874671626878336/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828852 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828827.222832:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626878336/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828852 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828827.222838:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828827.222840:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828827.222843:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626878336:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828827.222846:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626878464:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828827.222860:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828827.222867:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828827/real 1787828827]  req@ffff8cd0c3daad00 x1874671626878464/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828852 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828827.222875:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daad00 x1874671626878464/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828852 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828827.222880:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828827.222882:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828827.222884:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626878464:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828827.414202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828827.798287:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828828.438243:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828828.822200:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828829.462202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828829.846244:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828830.486165:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828830.870325:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828831.510261:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828831.894281:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828832.342154:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828832.342162:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828832.342165:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828832.342173:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2718
00000100:00080000:2.0:1787828832.342200:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828832.342206:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828832.342250:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daad00 x1874671626878592/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828832.342274:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828832.342277:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828832.342278:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828832.342281:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2718
00000100:00080000:2.0:1787828832.342283:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828832.342285:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828832.342293:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828832.342310:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626878720/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828832.342318:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828832.342320:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828832.342322:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828832.342324:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2718
00000100:00080000:2.0:1787828832.342326:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828832.342329:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828832.342332:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828832.342338:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626878848/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828832.342345:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787828832.342352:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626878592:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828832.342355:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828832.342359:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828832.342375:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2718
00000100:00080000:2.0:1787828832.342378:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828832.342381:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828832.342384:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828832.342394:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626878976/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828832.342435:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828832.342493:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828832/real 1787828832]  req@ffff8cd0c3daad00 x1874671626878592/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828857 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828832.342522:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daad00 x1874671626878592/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828857 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828832.342534:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828832.342538:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828832.342543:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626878592:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828832.342551:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626878720:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828832.342579:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828832.342594:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828832/real 1787828832]  req@ffff8cd0c3daa1c0 x1874671626878720/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828857 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828832.342606:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626878720/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828857 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828832.342614:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828832.342618:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828832.342622:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626878720:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828832.342647:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626878848:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828832.342671:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828832.342685:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828832/real 1787828832]  req@ffff8cd0c3da92c0 x1874671626878848/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828857 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828832.342698:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626878848/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828857 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828832.342737:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828832.342742:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828832.342757:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626878848:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828832.342764:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626878976:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828832.342789:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828832.342803:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828832/real 1787828832]  req@ffff8cd0c3da9e00 x1874671626878976/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828857 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828832.342816:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626878976/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828857 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828832.342825:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828832.342828:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828832.342832:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626878976:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828832.534137:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828832.918310:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828833.558156:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828833.942251:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828834.582145:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828834.966259:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828835.606132:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828835.990236:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828836.630136:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828837.014215:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828837.462241:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828837.462252:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828837.462257:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828837.462269:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2723
00000100:00080000:0.0:1787828837.462296:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828837.462304:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828837.462343:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626879104/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828837.462373:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828837.462377:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828837.462380:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828837.462383:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2723
00000100:00080000:0.0:1787828837.462387:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828837.462391:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828837.462416:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828837.462422:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626879104:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787828837.462435:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626879232/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828837.462450:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828837.462455:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828837.462457:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828837.462462:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2723
00000100:00080000:0.0:1787828837.462467:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828837.462472:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828837.462478:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787828837.462485:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787828837.462493:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626879360/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828837.462505:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828837.462509:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828837.462512:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828837.462516:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2723
00000100:00080000:0.0:1787828837.462521:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828837.462526:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828837.462532:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828837.462545:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626879488/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828837.462549:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828837/real 1787828837]  req@ffff8cd0c8e9e580 x1874671626879104/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828862 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828837.462587:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626879104/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828862 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828837.462606:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828837.462611:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828837.462615:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626879104:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828837.462639:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626879232:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828837.462682:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828837.462699:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828837/real 1787828837]  req@ffff8cd0c8e9fc00 x1874671626879232/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828862 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828837.462712:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626879232/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828862 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828837.462721:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828837.462725:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828837.462729:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626879232:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828837.462736:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626879360:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828837.462758:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828837.462770:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828837/real 1787828837]  req@ffff8cd0c8e9cf00 x1874671626879360/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828862 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828837.462783:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626879360/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828862 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828837.462792:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828837.462795:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828837.462798:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626879360:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828837.462815:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626879488:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828837.462839:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828837.462852:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828837/real 1787828837]  req@ffff8cd0c8e9de00 x1874671626879488/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828862 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828837.462865:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626879488/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828862 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828837.462873:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828837.462876:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828837.462880:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626879488:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828837.654139:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828838.038236:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828838.678177:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828839.062188:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828839.702195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828840.086193:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828840.726159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828841.110234:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828841.750146:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828842.134279:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828842.582174:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828842.582183:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828842.582187:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828842.582195:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2728
00000100:00080000:0.0:1787828842.582204:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828842.582210:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828842.582244:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626879616/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828842.582273:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828842.582277:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828842.582279:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828842.582283:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2728
00000100:00080000:0.0:1787828842.582287:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828842.582293:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828842.582304:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828842.582320:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626879744/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828842.582331:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626879616:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828842.582334:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828842.582339:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828842.582340:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828842.582344:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2728
00000100:00080000:0.0:1787828842.582347:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828842.582351:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828842.582356:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828842.582380:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626879872/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828842.582388:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:3.0:1787828842.582389:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787828842.582394:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828842.582396:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828842.582401:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2728
00000100:00080000:0.0:1787828842.582405:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828842.582409:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828842.582416:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828842.582430:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626880000/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828842.582448:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828842/real 1787828842]  req@ffff8cd0c8e9e940 x1874671626879616/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828867 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828842.582465:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626879616/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828867 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828842.582474:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828842.582477:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828842.582481:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626879616:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828842.582509:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626879744:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828842.582528:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828842.582538:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828842/real 1787828842]  req@ffff8cd0c8e9d680 x1874671626879744/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828867 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828842.582546:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626879744/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828867 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828842.582552:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828842.582554:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828842.582557:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626879744:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828842.582561:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626879872:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828842.582573:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828842.582579:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828842/real 1787828842]  req@ffff8cd0c8e9c3c0 x1874671626879872/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828867 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828842.582587:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626879872/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828867 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828842.582593:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828842.582594:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828842.582596:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626879872:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828842.582601:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626880000:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828842.582611:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828842.582617:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828842/real 1787828842]  req@ffff8cd0c8e9ed00 x1874671626880000/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828867 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828842.582648:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626880000/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828867 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828842.582655:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828842.582657:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828842.582659:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626880000:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828842.774214:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828843.158228:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828843.798175:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828844.182230:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828844.822160:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828845.206215:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828845.846194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828846.230261:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828846.870264:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828847.254248:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828847.702217:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828847.702235:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828847.702239:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828847.702251:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2733
00000100:00080000:0.0:1787828847.702263:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828847.702271:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828847.702315:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626880128/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828847.702354:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828847.702357:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828847.702377:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828847.702380:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2733
00000100:00080000:0.0:1787828847.702383:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828847.702387:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828847.702403:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828847.702418:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626880256/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828847.702430:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828847.702435:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828847.702437:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828847.702442:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2733
00000100:00080000:0.0:1787828847.702446:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828847.702457:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828847.702464:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626880128:192.168.203.145@tcp:250:kworker.0
00000008:00100000:0.0:1787828847.702465:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828847.702480:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d2c0 x1874671626880384/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828847.702489:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828847.702491:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828847.702493:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828847.702495:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2733
00000100:00080000:0.0:1787828847.702497:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828847.702500:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828847.702504:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828847.702530:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626880512/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828847.702589:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626880256:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828847.702652:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626880384:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828847.702676:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626880512:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828847.894200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828848.278278:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828848.918199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828849.302253:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828849.942187:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828850.326217:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:2.0:1787828850.774328:0:2346:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:2.0:1787828850.774367:0:2346:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:2.0:1787828850.774382:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787828850.774417:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787828850.774427:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787828850.774436:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:3.0:1787828850.774480:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828847/real 1787828850]  req@ffff8cd0c8e9d2c0 x1874671626880384/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828872 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828850.774519:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d2c0 x1874671626880384/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828872 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828850.774535:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828850.774555:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828850.774561:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626880384:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828850.774571:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828847/real 1787828850]  req@ffff8cd0c8e9f480 x1874671626880512/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828872 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828850.774607:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626880512/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828872 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828850.774618:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828850.774623:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828850.774628:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626880512:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828850.774636:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828847/real 1787828850]  req@ffff8cd0c8e9f840 x1874671626880128/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828872 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828850.774651:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626880128/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828872 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828850.774659:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828850.774663:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828850.774667:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626880128:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828850.774674:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828847/real 1787828850]  req@ffff8cd0c8e9c000 x1874671626880256/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828872 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828850.774686:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626880256/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828872 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828850.774695:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828850.774709:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828850.774713:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626880256:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828850.966206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828851.350280:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828851.990192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828852.374252:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828852.822212:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828852.822224:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828852.822229:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828852.822242:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2738
00000100:00080000:0.0:1787828852.822258:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828852.822269:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828852.822332:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626880640/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828852.822379:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828852.822382:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828852.822384:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828852.822388:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2738
00000100:00080000:0.0:1787828852.822391:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828852.822395:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828852.822417:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828852.822431:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626880768/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828852.822441:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828852.822446:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828852.822449:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1787828852.822460:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626880640:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828852.822478:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2738
00000100:00080000:0.0:1787828852.822499:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828852.822505:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828852.822512:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828852.822527:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626880896/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828852.822540:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828852.822544:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828852.822546:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828852.822550:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2738
00000100:00080000:0.0:1787828852.822554:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828852.822572:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828852.822579:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828852.822590:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626881024/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828852.822599:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828852.822673:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828852/real 1787828852]  req@ffff8cd0c8e9cb40 x1874671626880640/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828877 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828852.822698:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626880640/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828877 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828852.822712:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828852.822718:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828852.822723:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626880640:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828852.822732:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626880768:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828852.822762:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828852.822778:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828852/real 1787828852]  req@ffff8cd0c8e9c000 x1874671626880768/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828877 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828852.822792:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626880768/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828877 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828852.822801:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828852.822805:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828852.822808:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626880768:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828852.822850:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626880896:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828852.822878:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828852.822890:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828852/real 1787828852]  req@ffff8cd0c8e9f840 x1874671626880896/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828877 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828852.822903:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626880896/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828877 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828852.822912:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828852.822916:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828852.822920:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626880896:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828852.822927:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626881024:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828852.822949:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828852.822962:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828852/real 1787828852]  req@ffff8cd0c8e9f480 x1874671626881024/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828877 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828852.822975:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626881024/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828877 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828852.822985:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828852.822988:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828852.822992:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626881024:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828853.014158:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828853.398248:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828854.038204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828854.422314:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828855.062209:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828855.446186:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828856.086144:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828856.470266:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828857.110172:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828857.494182:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828857.942214:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828857.942221:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828857.942225:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828857.942233:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2743
00000100:00080000:2.0:1787828857.942242:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828857.942248:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828857.942280:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626881152/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828857.942303:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828857.942322:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828857.942337:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828857.942341:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2743
00000100:00080000:2.0:1787828857.942344:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828857.942347:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828857.942360:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828857.942376:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626881280/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828857.942389:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828857.942395:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:3.0:1787828857.942395:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626881152:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828857.942401:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828857.942410:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2743
00000100:00080000:2.0:1787828857.942414:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828857.942418:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828857.942425:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828857.942437:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626881408/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828857.942445:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828857.942449:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828857.942451:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828857.942454:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2743
00000100:00080000:2.0:1787828857.942469:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828857.942473:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828857.942477:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787828857.942493:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828857.942512:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626881536/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828857.942563:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828857/real 1787828857]  req@ffff8cd0c3daa580 x1874671626881152/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828882 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828857.942581:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626881152/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828882 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828857.942590:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828857.942594:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828857.942597:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626881152:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828857.942624:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626881280:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828857.942643:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828857.942652:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828857/real 1787828857]  req@ffff8cd0c3dabc00 x1874671626881280/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828882 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828857.942660:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626881280/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828882 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828857.942666:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828857.942668:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828857.942670:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626881280:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828857.942674:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626881408:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828857.942687:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828857.942693:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828857/real 1787828857]  req@ffff8cd0c3da8000 x1874671626881408/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828882 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828857.942714:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626881408/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828882 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828857.942720:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828857.942722:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828857.942724:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626881408:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828857.942728:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626881536:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828857.942741:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828857.942748:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828857/real 1787828857]  req@ffff8cd0c3dab840 x1874671626881536/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828882 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828857.942755:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626881536/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828882 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828857.942760:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828857.942762:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828857.942763:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626881536:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828858.134260:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828858.518295:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828859.158197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787828859.542229:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828860.182161:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828860.566196:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828861.206172:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828861.590216:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828862.230180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828862.614264:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828863.062229:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828863.062244:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828863.062249:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828863.062262:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2748
00000100:00080000:0.0:1787828863.062275:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828863.062290:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828863.062351:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d2c0 x1874671626881664/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828863.062379:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828863.062382:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828863.062383:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828863.062386:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2748
00000100:00080000:0.0:1787828863.062390:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828863.062394:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828863.062403:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828863.062415:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626881792/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828863.062416:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626881664:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828863.062432:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828863.062435:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828863.062437:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828863.062439:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2748
00000100:00080000:0.0:1787828863.062442:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828863.062445:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828863.062448:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828863.062457:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626881920/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828863.062468:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828863.062473:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828863.062475:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828863.062486:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2748
00000100:00080000:0.0:1787828863.062506:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1787828863.062508:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787828863.062514:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828863.062523:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828863.062537:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626882048/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828863.062568:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828863/real 1787828863]  req@ffff8cd0c8e9d2c0 x1874671626881664/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828888 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828863.062598:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d2c0 x1874671626881664/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828888 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828863.062612:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828863.062617:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828863.062621:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626881664:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828863.062646:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626881792:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828863.062667:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828863.062677:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828863/real 1787828863]  req@ffff8cd0c8e9ed00 x1874671626881792/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828888 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828863.062686:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626881792/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828888 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828863.062692:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828863.062695:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828863.062697:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626881792:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828863.062701:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626881920:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828863.062713:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828863.062720:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828863/real 1787828863]  req@ffff8cd0c8e9c3c0 x1874671626881920/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828888 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828863.062728:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626881920/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828888 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828863.062743:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828863.062745:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828863.062748:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626881920:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828863.062752:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626882048:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828863.062766:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828863.062773:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828863/real 1787828863]  req@ffff8cd0c8e9d680 x1874671626882048/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828888 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828863.062780:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626882048/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828888 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828863.062786:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828863.062788:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828863.062790:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626882048:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828863.254203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828863.638297:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828864.278229:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828864.662359:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828865.302197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828865.686212:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828866.326211:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828866.710300:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828867.350195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828867.734196:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828868.182199:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828868.182210:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828868.182215:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828868.182223:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2754
00000100:00080000:2.0:1787828868.182232:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828868.182238:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828868.182288:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626882176/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828868.182312:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828868.182315:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828868.182316:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828868.182319:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2754
00000100:00080000:2.0:1787828868.182321:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828868.182324:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828868.182349:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828868.182358:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626882304/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828868.182369:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787828868.182379:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626882176:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828868.182388:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828868.182390:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828868.182394:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2754
00000100:00080000:2.0:1787828868.182398:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828868.182401:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828868.182406:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828868.182428:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626882432/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828868.182436:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787828868.182439:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828868.182443:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828868.182444:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828868.182449:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2754
00000100:00080000:2.0:1787828868.182465:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828868.182471:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828868.182475:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828868.182502:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626882560/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828868.182518:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828868/real 1787828868]  req@ffff8cd0c3dab0c0 x1874671626882176/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828893 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828868.182534:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626882176/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828893 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828868.182548:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828868.182551:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828868.182555:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626882176:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828868.182575:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626882304:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828868.182594:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828868.182604:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828868/real 1787828868]  req@ffff8cd0c3da9a40 x1874671626882304/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828893 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828868.182612:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626882304/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828893 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828868.182618:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828868.182620:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828868.182622:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626882304:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828868.182626:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626882432:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828868.182638:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828868.182644:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828868/real 1787828868]  req@ffff8cd0c3da8780 x1874671626882432/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828893 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828868.182651:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626882432/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828893 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828868.182656:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828868.182658:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828868.182660:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626882432:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828868.182669:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626882560:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828868.182682:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828868.182689:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828868/real 1787828868]  req@ffff8cd0c3daa940 x1874671626882560/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828893 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828868.182696:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626882560/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828893 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828868.182702:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828868.182704:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828868.182705:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626882560:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828868.374198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828868.758228:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828869.398144:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828869.782237:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828870.422237:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828870.806263:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828871.446147:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828871.830323:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828872.470180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828872.854247:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828873.302182:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828873.302190:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828873.302193:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828873.302201:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2759
00000100:00080000:1.0:1787828873.302211:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828873.302217:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828873.302249:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626882688/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828873.302271:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828873.302274:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828873.302275:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828873.302281:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2759
00000100:00080000:1.0:1787828873.302284:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828873.302287:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828873.302295:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828873.302319:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626882816/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828873.302328:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828873.302332:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:3.0:1787828873.302332:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626882688:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787828873.302336:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828873.302343:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2759
00000100:00080000:1.0:1787828873.302346:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828873.302349:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828873.302353:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828873.302363:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626882944/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828873.302371:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828873.302373:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828873.302375:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828873.302379:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2759
00000100:00080000:1.0:1787828873.302384:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828873.302388:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828873.302393:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787828873.302398:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787828873.302410:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626883072/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828873.302452:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828873/real 1787828873]  req@ffff8cd0d48f7c00 x1874671626882688/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828898 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828873.302467:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626882688/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828898 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828873.302476:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828873.302479:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828873.302488:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626882688:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828873.302513:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626882816:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828873.302536:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828873.302545:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828873/real 1787828873]  req@ffff8cd0d48f52c0 x1874671626882816/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828898 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828873.302553:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626882816/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828898 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828873.302559:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828873.302561:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828873.302563:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626882816:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828873.302567:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626882944:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828873.302579:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828873.302585:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828873/real 1787828873]  req@ffff8cd0d48f6d00 x1874671626882944/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828898 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828873.302592:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626882944/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828898 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828873.302597:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828873.302598:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828873.302600:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626882944:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828873.302604:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626883072:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828873.302613:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828873.302626:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828873/real 1787828873]  req@ffff8cd0d48f61c0 x1874671626883072/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828898 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828873.302634:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626883072/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828898 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828873.302639:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828873.302641:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828873.302643:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626883072:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828873.494185:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828873.878318:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828874.518208:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828874.902232:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828875.542158:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828875.926321:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828876.566152:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828876.950267:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828877.590151:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828877.974241:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828878.422249:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828878.422260:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828878.422309:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828878.422325:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2764
00000100:00080000:1.0:1787828878.422342:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828878.422352:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828878.422452:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626883200/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828878.422485:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828878.422488:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828878.422490:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828878.422494:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2764
00000100:00080000:1.0:1787828878.422498:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828878.422503:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828878.422518:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828878.422538:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626883328/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828878.422547:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626883200:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787828878.422566:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828878.422572:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787828878.422574:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828878.422582:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2764
00000100:00080000:1.0:1787828878.422588:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828878.422594:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828878.422603:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828878.422620:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626883456/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828878.422648:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828878.422656:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000800:00000100:3.0:1787828878.422656:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787828878.422661:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828878.422667:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2764
00000100:00080000:1.0:1787828878.422674:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828878.422678:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828878.422685:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828878.422699:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626883584/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828878.422723:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828878/real 1787828878]  req@ffff8cd0d48f61c0 x1874671626883200/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828903 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828878.422744:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626883200/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828903 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828878.422755:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828878.422759:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828878.422780:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626883200:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828878.422819:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626883328:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828878.422844:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828878.422855:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828878/real 1787828878]  req@ffff8cd0d48f6d00 x1874671626883328/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828903 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828878.422866:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626883328/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828903 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828878.422873:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828878.422875:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828878.422878:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626883328:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828878.422883:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626883456:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828878.422898:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828878.422906:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828878/real 1787828878]  req@ffff8cd0d48f52c0 x1874671626883456/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828903 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828878.422915:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626883456/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828903 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828878.422922:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828878.422924:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828878.422926:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626883456:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828878.422931:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626883584:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828878.422943:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828878.422950:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828878/real 1787828878]  req@ffff8cd0d48f7c00 x1874671626883584/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828903 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828878.422967:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626883584/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828903 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828878.422974:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828878.422977:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828878.422979:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626883584:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828878.614203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828878.998278:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828879.638228:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828880.022339:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828880.662216:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828881.046314:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828881.686183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828882.070247:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828882.710185:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828883.094271:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828883.542162:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828883.542172:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828883.542176:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828883.542187:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2769
00000100:00080000:1.0:1787828883.542199:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828883.542205:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828883.542249:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626883712/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828883.542276:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828883.542279:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828883.542281:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828883.542300:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2769
00000100:00080000:1.0:1787828883.542304:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828883.542308:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828883.542319:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828883.542331:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626883840/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828883.542338:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626883712:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787828883.542359:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828883.542364:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787828883.542366:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828883.542371:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2769
00000100:00080000:1.0:1787828883.542374:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828883.542378:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828883.542383:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828883.542400:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626883968/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828883.542409:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828883.542412:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828883.542414:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828883.542417:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2769
00000100:00080000:1.0:1787828883.542420:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828883.542426:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:3.0:1787828883.542428:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:1.0:1787828883.542436:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828883.542451:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626884096/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828883.542485:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828883/real 1787828883]  req@ffff8cd0d48f70c0 x1874671626883712/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828908 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828883.542505:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626883712/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828908 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828883.542515:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828883.542519:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828883.542523:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626883712:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828883.542545:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626883840:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828883.542597:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828883.542614:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828883/real 1787828883]  req@ffff8cd0d48f43c0 x1874671626883840/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828908 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828883.542630:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626883840/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828908 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828883.542638:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828883.542641:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828883.542644:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626883840:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828883.542649:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626883968:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828883.542669:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828883.542678:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828883/real 1787828883]  req@ffff8cd0d48f5a40 x1874671626883968/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828908 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828883.542687:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626883968/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828908 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828883.542693:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828883.542695:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828883.542698:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626883968:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828883.542702:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626884096:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828883.542714:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828883.542721:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828883/real 1787828883]  req@ffff8cd0d48f7840 x1874671626884096/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828908 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828883.542741:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626884096/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828908 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828883.542747:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828883.542750:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828883.542752:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626884096:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828883.734206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828884.118255:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828884.758158:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828885.142212:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828885.782249:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828886.166247:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828886.806162:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828887.190349:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828887.830202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828888.214234:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828888.662192:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828888.662204:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828888.662209:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828888.662246:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2774
00000100:00080000:0.0:1787828888.662257:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828888.662265:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828888.662303:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626884224/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828888.662346:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828888.662350:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828888.662351:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828888.662355:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2774
00000100:00080000:0.0:1787828888.662359:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828888.662362:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828888.662373:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828888.662388:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626884352/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828888.662400:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787828888.662405:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626884224:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828888.662419:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828888.662421:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828888.662426:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2774
00000100:00080000:0.0:1787828888.662430:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828888.662434:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828888.662440:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828888.662453:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626884480/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828888.662462:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828888.662466:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828888.662493:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828888.662497:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2774
00000100:00080000:0.0:1787828888.662501:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828888.662504:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:3.0:1787828888.662509:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:0.0:1787828888.662513:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828888.662526:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626884608/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828888.662586:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828888/real 1787828888]  req@ffff8cd0c8e9e940 x1874671626884224/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828913 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828888.662611:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626884224/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828913 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828888.662632:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828888.662638:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828888.662644:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626884224:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828888.662676:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626884352:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828888.662749:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828888.662767:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828888/real 1787828888]  req@ffff8cd0c8e9de00 x1874671626884352/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828913 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828888.662785:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626884352/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828913 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828888.662823:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828888.662828:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828888.662832:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626884352:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828888.662841:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626884480:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828888.662893:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828888.662907:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828888/real 1787828888]  req@ffff8cd0c8e9cf00 x1874671626884480/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828913 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828888.662921:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626884480/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828913 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828888.662931:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828888.662935:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828888.662939:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626884480:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828888.662945:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626884608:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828888.662969:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828888.662992:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828888/real 1787828888]  req@ffff8cd0c8e9fc00 x1874671626884608/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828913 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828888.663021:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626884608/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828913 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828888.663031:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828888.663035:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828888.663039:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626884608:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828888.854141:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828889.238228:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828889.878187:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828890.262267:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828890.902162:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828891.286333:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828891.926228:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828892.310278:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828892.950249:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828893.334320:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828893.782207:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828893.782217:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828893.782222:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828893.782237:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2779
00000100:00080000:2.0:1787828893.782249:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828893.782257:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828893.782320:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626884736/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828893.782343:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828893.782345:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828893.782347:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828893.782349:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2779
00000100:00080000:2.0:1787828893.782353:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828893.782355:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828893.782363:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828893.782373:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8b40 x1874671626884864/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828893.782380:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828893.782383:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828893.782385:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828893.782387:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2779
00000100:00080000:2.0:1787828893.782389:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1787828893.782394:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626884736:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828893.782395:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828893.782403:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828893.782412:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626884992/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828893.782419:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828893.782422:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828893.782423:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828893.782431:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2779
00000100:00080000:2.0:1787828893.782461:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828893.782467:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828893.782474:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787828893.782484:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828893.782491:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626885120/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828893.782563:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828893/real 1787828893]  req@ffff8cd0c3da8f00 x1874671626884736/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828918 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828893.782604:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626884736/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828918 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828893.782617:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828893.782640:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828893.782646:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626884736:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828893.782667:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626884864:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828893.782713:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828893.782747:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828893/real 1787828893]  req@ffff8cd0c3da8b40 x1874671626884864/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828918 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828893.782761:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8b40 x1874671626884864/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828918 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828893.782771:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828893.782774:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828893.782777:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626884864:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828893.782802:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626884992:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828893.782823:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828893.782834:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828893/real 1787828893]  req@ffff8cd0c3da83c0 x1874671626884992/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828918 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828893.782846:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626884992/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828918 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828893.782869:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828893.782871:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828893.782873:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626884992:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828893.782877:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626885120:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828893.782889:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828893.782895:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828893/real 1787828893]  req@ffff8cd0c3da9680 x1874671626885120/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828918 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828893.782913:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626885120/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828918 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828893.782918:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828893.782920:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828893.782922:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626885120:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828893.974326:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828894.358232:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828894.998201:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828895.382232:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828896.022184:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828896.406189:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828897.046166:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828897.430236:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828898.070167:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828898.454305:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828898.902144:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828898.902153:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828898.902157:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828898.902165:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2784
00000100:00080000:2.0:1787828898.902174:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828898.902180:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828898.902215:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626885248/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828898.902248:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828898.902252:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828898.902255:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828898.902258:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2784
00000100:00080000:2.0:1787828898.902261:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828898.902264:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828898.902272:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828898.902282:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626885376/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828898.902291:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787828898.902297:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626885248:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828898.902307:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828898.902309:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828898.902312:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2784
00000100:00080000:2.0:1787828898.902315:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828898.902319:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828898.902323:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828898.902332:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8b40 x1874671626885504/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828898.902340:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828898.902343:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828898.902344:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828898.902346:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2784
00000100:00080000:2.0:1787828898.902351:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828898.902354:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828898.902357:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828898.902363:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626885632/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828898.902367:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828898.902417:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828898/real 1787828898]  req@ffff8cd0c3da9680 x1874671626885248/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828923 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828898.902436:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626885248/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828923 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828898.902448:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828898.902452:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828898.902456:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626885248:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828898.902479:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626885376:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828898.902506:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828898.902516:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828898/real 1787828898]  req@ffff8cd0c3da83c0 x1874671626885376/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828923 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828898.902525:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626885376/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828923 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828898.902531:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828898.902534:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828898.902536:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626885376:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828898.902540:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626885504:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828898.902551:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828898.902560:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828898/real 1787828898]  req@ffff8cd0c3da8b40 x1874671626885504/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828923 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828898.902573:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8b40 x1874671626885504/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828923 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828898.902581:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828898.902584:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828898.902588:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626885504:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828898.902594:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626885632:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828898.902615:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828898.902624:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828898/real 1787828898]  req@ffff8cd0c3da8f00 x1874671626885632/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828923 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828898.902643:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626885632/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828923 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828898.902648:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828898.902650:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828898.902653:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626885632:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828899.094139:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828899.478267:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828900.118177:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828900.502249:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828901.142145:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828901.526214:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828902.166144:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828902.550258:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828903.190144:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828903.574226:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828904.022224:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828904.022236:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828904.022241:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828904.022251:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2789
00000100:00080000:3.0:1787828904.022262:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828904.022270:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828904.022319:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626885760/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828904.022347:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828904.022351:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828904.022353:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828904.022356:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2789
00000100:00080000:3.0:1787828904.022360:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828904.022380:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828904.022391:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828904.022404:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626885760:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828904.022408:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626885888/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828904.022430:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828904.022436:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828904.022439:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828904.022445:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2789
00000100:00080000:3.0:1787828904.022475:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828904.022483:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828904.022489:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787828904.022497:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828904.022535:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626886016/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828904.022546:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828904/real 1787828904]  req@ffff8cd0c8eae1c0 x1874671626885760/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828929 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828904.022562:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828904.022569:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:2.0:1787828904.022575:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626885760/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828929 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828904.022575:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828904.022588:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2789
00000100:00080000:2.0:1787828904.022603:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828904.022610:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00080000:3.0:1787828904.022613:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1787828904.022614:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626885760:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787828904.022625:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828904.022633:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828904.022649:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626885888:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828904.022651:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626886144/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787828904.022688:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828904.022707:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828904/real 1787828904]  req@ffff8cd0c8ead2c0 x1874671626885888/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828929 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828904.022720:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626885888/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828929 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828904.022727:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828904.022731:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828904.022734:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626885888:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828904.022742:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626886016:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828904.022761:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828904.022771:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828904/real 1787828904]  req@ffff8cd0c8eae940 x1874671626886016/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828929 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828904.022782:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626886016/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828929 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828904.022789:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828904.022791:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828904.022794:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626886016:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828904.022816:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626886144:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828904.022848:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828904.022859:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828904/real 1787828904]  req@ffff8cd0c8eaf840 x1874671626886144/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828929 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828904.022871:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626886144/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828929 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828904.022878:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828904.022881:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828904.022884:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626886144:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828904.214276:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828904.598240:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828905.238147:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828905.622227:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828906.262173:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828906.646250:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828907.286212:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828907.670295:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828908.310190:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828908.694242:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828909.142195:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828909.142203:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828909.142206:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828909.142215:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2794
00000100:00080000:0.0:1787828909.142225:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828909.142231:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828909.142265:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626886272/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828909.142301:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828909.142304:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828909.142306:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828909.142308:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2794
00000100:00080000:0.0:1787828909.142311:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828909.142314:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828909.142322:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828909.142333:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626886400/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828909.142341:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626886272:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828909.142343:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828909.142350:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828909.142351:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828909.142354:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2794
00000100:00080000:0.0:1787828909.142357:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828909.142359:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828909.142362:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828909.142369:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626886528/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828909.142376:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828909.142379:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828909.142380:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828909.142383:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2794
00000100:00080000:0.0:1787828909.142392:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828909.142395:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828909.142399:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787828909.142402:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787828909.142410:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626886656/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828909.142452:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828909/real 1787828909]  req@ffff8cd0c8e9e580 x1874671626886272/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828934 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828909.142473:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626886272/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828934 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828909.142482:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828909.142485:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828909.142488:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626886272:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828909.142507:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626886400:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828909.142539:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828909.142554:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828909/real 1787828909]  req@ffff8cd0c8e9fc00 x1874671626886400/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828934 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828909.142568:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626886400/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828934 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828909.142575:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828909.142577:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828909.142579:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626886400:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828909.142584:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626886528:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828909.142601:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828909.142608:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828909/real 1787828909]  req@ffff8cd0c8e9cf00 x1874671626886528/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828934 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828909.142615:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626886528/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828934 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828909.142621:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828909.142623:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828909.142625:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626886528:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828909.142629:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626886656:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828909.142639:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828909.142644:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828909/real 1787828909]  req@ffff8cd0c8e9de00 x1874671626886656/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828934 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828909.142669:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626886656/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828934 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828909.142677:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828909.142680:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828909.142684:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626886656:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828909.334228:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828909.718248:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828910.358150:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828910.742261:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828911.382209:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828911.766260:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828912.406215:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828912.790250:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828913.430184:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828913.814271:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828914.262185:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828914.262196:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828914.262200:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828914.262210:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2800
00000100:00080000:0.0:1787828914.262222:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828914.262230:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828914.262269:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626886784/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828914.262299:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828914.262303:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828914.262305:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828914.262309:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2800
00000100:00080000:0.0:1787828914.262313:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828914.262317:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828914.262326:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828914.262341:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626886912/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828914.262345:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626886784:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828914.262361:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828914.262378:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828914.262380:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828914.262386:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2800
00000100:00080000:0.0:1787828914.262392:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828914.262397:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828914.262404:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828914.262416:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626887040/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828914.262428:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828914.262431:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828914.262434:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828914.262439:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2800
00000100:00080000:0.0:1787828914.262444:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1787828914.262447:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626886912:192.168.203.145@tcp:38:kworker.0
00000100:00080000:0.0:1787828914.262452:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828914.262460:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828914.262476:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626887168/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828914.262481:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626887040:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828914.262509:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626887168:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828914.454159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828914.838268:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828915.478192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828915.862279:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828916.502186:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828916.886296:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:2.0:1787828917.334414:0:2343:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:2.0:1787828917.334468:0:2343:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:2.0:1787828917.334498:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787828917.334543:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787828917.334554:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787828917.334564:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:3.0:1787828917.334607:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828914/real 1787828917]  req@ffff8cd0c8e9ed00 x1874671626887168/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828939 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828917.334637:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626887168/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828939 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828917.334650:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828917.334656:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828917.334661:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626887168:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828917.334671:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828914/real 1787828917]  req@ffff8cd0c8e9d680 x1874671626886912/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828939 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828917.334685:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626886912/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828939 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828917.334694:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828917.334699:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828917.334703:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626886912:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828917.334730:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828914/real 1787828917]  req@ffff8cd0c8e9c3c0 x1874671626887040/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828939 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828917.334746:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626887040/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828939 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828917.334755:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828917.334759:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828917.334763:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626887040:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828917.334770:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828914/real 1787828917]  req@ffff8cd0c8e9e940 x1874671626886784/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828939 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828917.334783:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626886784/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828939 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828917.334792:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828917.334795:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828917.334799:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626886784:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828917.526205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828917.910217:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828918.550196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828918.934296:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828919.382218:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828919.382230:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828919.382250:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828919.382262:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2805
00000100:00080000:2.0:1787828919.382275:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828919.382285:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828919.382338:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626887296/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828919.382375:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828919.382380:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828919.382382:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828919.382388:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2805
00000100:00080000:2.0:1787828919.382393:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828919.382399:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828919.382411:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828919.382433:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626887424/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828919.382433:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626887296:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787828919.382456:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828919.382461:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828919.382464:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828919.382471:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2805
00000100:00080000:2.0:1787828919.382476:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828919.382482:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828919.382489:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828919.382505:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626887552/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828919.382514:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787828919.382545:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828919.382550:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828919.382553:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828919.382560:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2805
00000100:00080000:2.0:1787828919.382566:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1787828919.382569:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828919/real 1787828919]  req@ffff8cd0c3daa940 x1874671626887296/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828944 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828919.382576:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828919.382588:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828919.382604:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626887296/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828944 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828919.382610:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626887680/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828919.382659:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828919.382666:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828919.382672:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626887296:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828919.382702:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626887424:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828919.382746:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828919.382765:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828919/real 1787828919]  req@ffff8cd0c3da8780 x1874671626887424/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828944 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828919.382783:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626887424/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828944 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828919.382794:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828919.382799:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828919.382803:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626887424:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828919.382811:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626887552:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828919.382840:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828919.382855:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828919/real 1787828919]  req@ffff8cd0c3da9a40 x1874671626887552/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828944 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828919.382870:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626887552/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828944 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828919.382881:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828919.382885:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828919.382889:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626887552:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828919.382907:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626887680:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828919.382935:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828919.382949:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828919/real 1787828919]  req@ffff8cd0c3dab0c0 x1874671626887680/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828944 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828919.382963:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626887680/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828944 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828919.382972:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828919.382975:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828919.382979:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626887680:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828919.574199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828919.958225:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828920.598158:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828920.982246:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828921.622207:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828922.006290:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828922.646154:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828923.030274:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828923.670265:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828924.054283:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828924.502264:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828924.502278:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828924.502285:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828924.502300:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2810
00000100:00080000:3.0:1787828924.502341:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828924.502352:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828924.502398:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626887808/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828924.502436:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828924.502440:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828924.502460:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828924.502466:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2810
00000100:00080000:3.0:1787828924.502473:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828924.502479:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828924.502492:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828924.502495:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626887808:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828924.502512:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626887936/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828924.502529:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828924.502533:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828924.502536:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828924.502540:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2810
00000100:00080000:3.0:1787828924.502544:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828924.502549:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828924.502555:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828924.502569:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626888064/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828924.502581:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828924.502585:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828924.502591:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1787828924.502593:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787828924.502600:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2810
00000100:00080000:3.0:1787828924.502607:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828924.502613:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828924.502618:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828924.502644:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626888192/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828924.502678:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828924/real 1787828924]  req@ffff8cd0c8eac000 x1874671626887808/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828949 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828924.502754:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626887808/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828949 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828924.502769:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828924.502775:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828924.502784:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626887808:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828924.502818:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626887936:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828924.502855:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828924.502914:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828924/real 1787828924]  req@ffff8cd0c8eacf00 x1874671626887936/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828949 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828924.502940:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626887936/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828949 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828924.502957:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828924.502964:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828924.502987:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626887936:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828924.502998:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626888064:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828924.503031:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828924.503047:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828924/real 1787828924]  req@ffff8cd0c8eaf840 x1874671626888064/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828949 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828924.503130:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626888064/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828949 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828924.503142:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828924.503146:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828924.503151:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626888064:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828924.503158:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626888192:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828924.503186:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828924.503200:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828924/real 1787828924]  req@ffff8cd0c8eae940 x1874671626888192/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828949 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828924.503214:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626888192/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828949 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828924.503234:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828924.503238:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828924.503242:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626888192:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828924.694227:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828925.078225:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828925.718202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828926.102406:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828926.742217:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828927.126236:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828927.766156:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828928.150225:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828928.790261:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828929.174229:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828929.622180:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828929.622188:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828929.622191:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828929.622199:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2815
00000100:00080000:1.0:1787828929.622208:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828929.622214:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828929.622251:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626888320/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828929.622295:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828929.622298:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828929.622299:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828929.622303:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2815
00000100:00080000:1.0:1787828929.622306:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828929.622309:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828929.622318:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828929.622329:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626888448/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828929.622339:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1787828929.622339:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626888320:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787828929.622348:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787828929.622350:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828929.622354:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2815
00000100:00080000:1.0:1787828929.622356:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828929.622358:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828929.622362:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828929.622370:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626888576/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828929.622377:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828929.622379:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828929.622381:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828929.622383:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2815
00000100:00080000:1.0:1787828929.622393:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828929.622396:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828929.622400:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787828929.622402:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787828929.622412:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626888704/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828929.622439:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828929/real 1787828929]  req@ffff8cd0d48f4b40 x1874671626888320/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828954 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828929.622453:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626888320/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828954 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828929.622471:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828929.622474:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828929.622477:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626888320:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828929.622497:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626888448:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828929.622512:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828929.622521:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828929/real 1787828929]  req@ffff8cd0d48f6580 x1874671626888448/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828954 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828929.622538:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626888448/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828954 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828929.622554:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828929.622556:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828929.622558:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626888448:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828929.622562:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626888576:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828929.622577:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828929.622584:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828929/real 1787828929]  req@ffff8cd0d48f7480 x1874671626888576/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828954 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828929.622603:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626888576/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828954 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828929.622608:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828929.622610:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828929.622612:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626888576:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828929.622615:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626888704:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828929.622626:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828929.622632:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828929/real 1787828929]  req@ffff8cd0d48f4000 x1874671626888704/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828954 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828929.622639:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626888704/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828954 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828929.622643:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828929.622645:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828929.622652:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626888704:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828929.814148:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828930.198201:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828930.838154:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828931.222256:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828931.862157:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828932.246268:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828932.886210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828933.270262:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828933.910146:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828934.294275:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787828934.742341:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828934.742357:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787828934.742363:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828934.742394:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2820
00000100:00080000:3.0:1787828934.742418:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828934.742433:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828934.742528:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626888832/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828934.742587:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828934.742594:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828934.742597:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828934.742605:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2820
00000100:00080000:3.0:1787828934.742611:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828934.742630:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828934.742653:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626888832:192.168.203.145@tcp:250:kworker.0
00000008:00100000:3.0:1787828934.742680:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828934.742709:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626888960/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828934.742743:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828934.742747:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787828934.742749:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828934.742756:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2820
00000100:00080000:3.0:1787828934.742762:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787828934.742770:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:2.0:1787828934.742771:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:3.0:1787828934.742780:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828934.742797:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626889088/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828934.742811:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787828934.742816:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787828934.742818:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787828934.742824:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2820
00000100:00080000:3.0:1787828934.742828:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1787828934.742831:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828934/real 1787828934]  req@ffff8cd0c8ead2c0 x1874671626888832/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828959 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828934.742867:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787828934.742883:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828934.742897:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626888832/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828959 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828934.742911:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626889216/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828934.742922:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828934.742938:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828934.742950:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626888832:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828934.742984:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626888960:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828934.743062:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828934.743077:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828934/real 1787828934]  req@ffff8cd0c8eae1c0 x1874671626888960/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828959 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828934.743140:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626888960/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828959 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828934.743150:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828934.743153:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828934.743156:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626888960:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828934.743164:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626889088:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828934.743220:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828934.743236:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828934/real 1787828934]  req@ffff8cd0c8eada40 x1874671626889088/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828959 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828934.743251:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626889088/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828959 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828934.743262:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828934.743266:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828934.743270:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626889088:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828934.743278:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626889216:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828934.743303:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828934.743316:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828934/real 1787828934]  req@ffff8cd0c8eac780 x1874671626889216/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828959 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828934.743335:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626889216/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828959 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828934.743342:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828934.743344:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828934.743347:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626889216:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828934.934149:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828935.318308:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828935.958151:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828936.342286:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828936.982178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828937.366342:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828938.006171:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828938.390405:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828939.030233:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828939.414295:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828939.862290:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828939.862301:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828939.862306:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828939.862316:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2825
00000100:00080000:1.0:1787828939.862328:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828939.862335:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828939.862392:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626889344/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828939.862428:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828939.862433:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828939.862435:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828939.862440:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2825
00000100:00080000:1.0:1787828939.862445:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828939.862450:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828939.862465:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828939.862504:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626889344:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787828939.862509:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626889472/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828939.862532:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828939.862536:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787828939.862539:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828939.862544:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2825
00000100:00080000:1.0:1787828939.862551:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828939.862557:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828939.862563:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828939.862576:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626889600/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828939.862585:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828939.862588:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828939.862589:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828939.862593:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2825
00000100:00080000:1.0:1787828939.862599:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1787828939.862600:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787828939.862604:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828939.862607:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828939.862615:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626889728/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828939.862716:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828939/real 1787828939]  req@ffff8cd0d48f4000 x1874671626889344/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828939.862763:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626889344/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828939.862779:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828939.862786:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828939.862793:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626889344:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828939.862842:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626889472:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828939.862884:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828939.862906:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828939/real 1787828939]  req@ffff8cd0d48f7480 x1874671626889472/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828939.862926:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626889472/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828939.862942:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828939.862947:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828939.862953:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626889472:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828939.863007:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626889600:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828939.863037:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828939.863055:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828939/real 1787828939]  req@ffff8cd0d48f6580 x1874671626889600/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828939.863071:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626889600/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828939.863160:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828939.863167:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828939.863174:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626889600:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828939.863185:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626889728:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828939.863237:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828939.863255:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828939/real 1787828939]  req@ffff8cd0d48f4b40 x1874671626889728/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828939.863288:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626889728/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828939.863299:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828939.863304:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828939.863308:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626889728:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828940.054166:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828940.438211:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828941.078134:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828941.462237:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828942.102182:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828942.486305:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828943.126271:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828943.510313:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828944.150220:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828944.534368:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828944.982228:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828944.982239:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828944.982243:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828944.982255:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2830
00000100:00080000:0.0:1787828944.982276:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828944.982285:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828944.982332:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d2c0 x1874671626889856/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828944.982379:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828944.982382:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828944.982384:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828944.982387:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2830
00000100:00080000:0.0:1787828944.982391:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828944.982395:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828944.982406:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828944.982419:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626889984/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828944.982429:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828944.982433:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828944.982434:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828944.982437:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2830
00000100:00080000:0.0:1787828944.982440:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828944.982443:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828944.982449:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828944.982453:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626889856:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787828944.982462:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626890112/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828944.982474:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828944.982478:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828944.982480:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828944.982486:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2830
00000100:00080000:0.0:1787828944.982498:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828944.982504:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828944.982510:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828944.982521:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626890240/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787828944.982542:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828944.982588:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828944/real 1787828944]  req@ffff8cd0c8e9d2c0 x1874671626889856/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828944.982606:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d2c0 x1874671626889856/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828944.982616:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828944.982632:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828944.982635:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626889856:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828944.982642:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626889984:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828944.982660:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828944.982669:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828944/real 1787828944]  req@ffff8cd0c8e9f480 x1874671626889984/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828944.982677:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626889984/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828944.982683:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828944.982685:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828944.982687:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626889984:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828944.982708:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626890112:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828944.982720:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828944.982726:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828944/real 1787828944]  req@ffff8cd0c8e9f840 x1874671626890112/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828944.982733:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626890112/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828944.982738:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828944.982740:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828944.982742:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626890112:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828944.982746:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626890240:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828944.982764:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828944.982771:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828944/real 1787828944]  req@ffff8cd0c8e9c000 x1874671626890240/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828944.982778:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626890240/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828944.982783:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828944.982785:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828944.982787:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626890240:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828945.174167:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828945.558268:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828946.198178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828946.582214:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828947.222175:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828947.606212:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828948.246203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828948.630237:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828949.270144:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828949.654241:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828950.102172:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828950.102182:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828950.102187:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828950.102197:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2835
00000100:00080000:1.0:1787828950.102208:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828950.102214:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828950.102251:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626890368/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828950.102278:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828950.102283:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828950.102285:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828950.102289:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2835
00000100:00080000:1.0:1787828950.102293:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828950.102298:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828950.102307:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828950.102321:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626890496/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828950.102332:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828950.102337:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787828950.102339:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828950.102344:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2835
00000100:00080000:1.0:1787828950.102348:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:2.0:1787828950.102354:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626890368:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787828950.102355:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828950.102374:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828950.102391:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626890624/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828950.102404:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828950.102409:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828950.102412:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828950.102416:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2835
00000100:00080000:1.0:1787828950.102421:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:2.0:1787828950.102422:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787828950.102431:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828950.102438:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828950.102452:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626890752/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828950.102461:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828950/real 1787828950]  req@ffff8cd0d48f7840 x1874671626890368/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828975 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828950.102477:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626890368/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828975 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828950.102485:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828950.102488:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828950.102491:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626890368:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828950.102496:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626890496:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828950.102511:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828950.102528:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828950/real 1787828950]  req@ffff8cd0d48f5a40 x1874671626890496/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828975 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828950.102537:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626890496/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828975 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828950.102542:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828950.102544:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828950.102547:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626890496:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828950.102565:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626890624:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828950.102578:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828950.102584:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828950/real 1787828950]  req@ffff8cd0d48f43c0 x1874671626890624/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828975 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828950.102592:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626890624/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828975 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828950.102597:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828950.102599:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828950.102601:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626890624:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828950.102604:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626890752:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828950.102614:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828950.102619:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828950/real 1787828950]  req@ffff8cd0d48f70c0 x1874671626890752/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828975 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828950.102626:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626890752/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828975 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828950.102647:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828950.102649:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828950.102651:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626890752:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828950.294178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828950.678300:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828951.318160:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828951.702288:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828952.342192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828952.726271:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828953.366177:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828953.750282:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828954.390241:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828954.774286:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828955.222171:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828955.222182:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828955.222188:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828955.222200:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2841
00000100:00080000:1.0:1787828955.222215:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828955.222225:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828955.222274:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626890880/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828955.222301:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828955.222320:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828955.222322:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828955.222326:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2841
00000100:00080000:1.0:1787828955.222330:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828955.222334:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828955.222342:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626890880:192.168.203.145@tcp:250:kworker.0
00000008:00100000:1.0:1787828955.222345:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828955.222358:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626891008/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828955.222367:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828955.222370:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787828955.222371:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828955.222374:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2841
00000100:00080000:1.0:1787828955.222377:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828955.222379:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828955.222383:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828955.222393:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626891136/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828955.222404:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828955.222409:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828955.222411:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:2.0:1787828955.222417:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787828955.222418:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2841
00000100:00080000:1.0:1787828955.222442:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828955.222449:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828955.222459:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828955.222461:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828955/real 1787828955]  req@ffff8cd0d48f7c00 x1874671626890880/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828980 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787828955.222472:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626891264/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828955.222480:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626890880/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828980 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828955.222491:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828955.222495:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828955.222498:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626890880:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828955.222521:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626891008:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828955.222539:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828955.222553:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828955/real 1787828955]  req@ffff8cd0d48f52c0 x1874671626891008/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828980 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828955.222567:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626891008/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828980 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828955.222577:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828955.222598:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828955.222602:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626891008:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828955.222609:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626891136:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828955.222628:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828955.222637:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828955/real 1787828955]  req@ffff8cd0d48f6d00 x1874671626891136/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828980 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828955.222652:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626891136/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828980 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828955.222658:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828955.222660:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828955.222663:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626891136:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828955.222667:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626891264:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828955.222679:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828955.222686:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828955/real 1787828955]  req@ffff8cd0d48f61c0 x1874671626891264/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828980 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828955.222693:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626891264/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828980 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828955.222699:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828955.222701:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828955.222702:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626891264:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828955.414250:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828955.798281:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828956.438199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828956.822218:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828957.462177:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828957.846298:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828958.486202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828958.870231:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828959.510161:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828959.894261:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828960.342213:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828960.342227:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828960.342233:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828960.342247:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2846
00000100:00080000:1.0:1787828960.342270:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828960.342279:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828960.342327:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626891392/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828960.342355:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828960.342357:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828960.342359:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828960.342362:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2846
00000100:00080000:1.0:1787828960.342365:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828960.342367:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828960.342377:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828960.342387:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626891520/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828960.342395:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828960.342398:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787828960.342416:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828960.342420:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2846
00000100:00080000:1.0:1787828960.342424:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828960.342428:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828960.342436:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626891392:192.168.203.145@tcp:250:kworker.0
00000008:00100000:1.0:1787828960.342437:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828960.342456:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626891648/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828960.342467:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828960.342470:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828960.342472:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828960.342476:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2846
00000100:00080000:1.0:1787828960.342478:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828960.342481:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828960.342485:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828960.342496:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626891776/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787828960.342521:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828960.342583:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828960/real 1787828960]  req@ffff8cd0d48f61c0 x1874671626891392/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828985 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828960.342604:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626891392/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828985 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828960.342613:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828960.342618:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828960.342622:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626891392:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828960.342631:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626891520:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828960.342657:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828960.342669:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828960/real 1787828960]  req@ffff8cd0d48f6d00 x1874671626891520/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828985 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828960.342678:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626891520/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828985 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828960.342684:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828960.342686:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828960.342688:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626891520:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828960.342709:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626891648:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828960.342724:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828960.342746:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828960/real 1787828960]  req@ffff8cd0d48f52c0 x1874671626891648/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828985 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828960.342754:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626891648/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828985 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828960.342759:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828960.342761:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828960.342763:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626891648:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828960.342767:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626891776:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828960.342780:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828960.342785:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828960/real 1787828960]  req@ffff8cd0d48f7c00 x1874671626891776/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828985 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828960.342792:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626891776/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828985 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828960.342797:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828960.342799:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828960.342801:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626891776:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828960.534178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828960.918235:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828961.558147:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828961.942318:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828962.582167:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828962.966298:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828963.606162:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828963.990276:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828964.630226:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828965.014331:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787828965.462223:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828965.462237:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787828965.462243:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828965.462254:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2851
00000100:00080000:1.0:1787828965.462269:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828965.462278:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787828965.462348:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626891904/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828965.462390:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828965.462397:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828965.462399:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828965.462403:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2851
00000100:00080000:1.0:1787828965.462408:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828965.462413:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828965.462426:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828965.462439:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626892032/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828965.462450:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828965.462457:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:2.0:1787828965.462459:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626891904:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787828965.462460:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828965.462471:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2851
00000100:00080000:1.0:1787828965.462475:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828965.462478:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828965.462483:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828965.462496:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626892160/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787828965.462506:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787828965.462509:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787828965.462511:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787828965.462522:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2851
00000100:00080000:1.0:1787828965.462543:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787828965.462547:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787828965.462551:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787828965.462561:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626892288/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787828965.462570:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828965.462625:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828965/real 1787828965]  req@ffff8cd0d48f70c0 x1874671626891904/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828990 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828965.462652:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626891904/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828990 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828965.462678:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828965.462682:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828965.462687:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626891904:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828965.462716:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626892032:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828965.462742:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828965.462755:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828965/real 1787828965]  req@ffff8cd0d48f43c0 x1874671626892032/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828990 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828965.462766:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626892032/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828990 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828965.462775:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828965.462798:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828965.462802:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626892032:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828965.462808:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626892160:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828965.462844:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828965.462852:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828965/real 1787828965]  req@ffff8cd0d48f5a40 x1874671626892160/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828990 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828965.462860:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626892160/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828990 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828965.462875:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828965.462877:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828965.462880:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626892160:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828965.462884:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626892288:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828965.462898:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828965.462905:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828965/real 1787828965]  req@ffff8cd0d48f7840 x1874671626892288/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828990 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828965.462913:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626892288/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828990 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828965.462918:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828965.462919:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828965.462921:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626892288:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828965.654150:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828966.038305:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828966.678166:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828967.062227:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828967.702166:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828968.086225:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828968.726141:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828969.110198:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828969.750143:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828970.134339:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828970.582234:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828970.582251:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828970.582258:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828970.582273:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2856
00000100:00080000:0.0:1787828970.582288:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828970.582299:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828970.582347:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626892416/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828970.582375:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828970.582381:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828970.582384:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828970.582390:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2856
00000100:00080000:0.0:1787828970.582395:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828970.582403:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828970.582411:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626892416:192.168.203.145@tcp:250:kworker.0
00000008:00100000:0.0:1787828970.582437:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828970.582478:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626892544/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828970.582497:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:2.0:1787828970.582500:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787828970.582532:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828970.582535:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828970.582541:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2856
00000100:00080000:0.0:1787828970.582546:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828970.582555:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828970.582558:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828970/real 1787828970]  req@ffff8cd0c8e9cb40 x1874671626892416/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828995 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:0.0:1787828970.582563:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828970.582587:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626892672/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828970.582599:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626892416/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828995 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787828970.582600:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828970.582614:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828970.582620:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828970.582634:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2856
00000100:00080000:0.0:1787828970.582637:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828970.582641:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828970.582653:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00080000:2.0:1787828970.582671:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:0.0:1787828970.582676:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626892800/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828970.582682:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828970.582698:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626892416:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828970.582737:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626892544:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828970.582776:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828970.582787:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828970/real 1787828970]  req@ffff8cd0c8e9c000 x1874671626892544/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828995 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828970.582795:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626892544/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828995 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828970.582801:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828970.582804:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828970.582806:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626892544:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828970.582810:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626892672:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828970.582823:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828970.582829:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828970/real 1787828970]  req@ffff8cd0c8e9f840 x1874671626892672/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828995 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828970.582836:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626892672/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828995 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828970.582841:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828970.582843:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828970.582845:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626892672:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828970.582855:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626892800:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828970.582870:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828970.582877:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828970/real 1787828970]  req@ffff8cd0c8e9f480 x1874671626892800/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828995 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828970.582884:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626892800/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828995 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828970.582889:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828970.582891:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828970.582893:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626892800:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828970.774168:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828971.158254:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828971.798281:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828972.182224:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828972.822181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828973.206258:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828973.846169:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828974.230274:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828974.870225:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828975.254337:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828975.702183:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828975.702191:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828975.702194:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828975.702202:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2861
00000100:00080000:0.0:1787828975.702212:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828975.702218:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828975.702251:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d2c0 x1874671626892928/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828975.702290:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828975.702293:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828975.702294:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828975.702297:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2861
00000100:00080000:0.0:1787828975.702300:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828975.702302:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828975.702311:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828975.702320:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626893056/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828975.702328:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828975.702330:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828975.702332:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828975.702334:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2861
00000100:00080000:0.0:1787828975.702336:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828975.702338:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828975.702342:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828975.702348:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626892928:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787828975.702349:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626893184/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828975.702364:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828975.702367:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828975.702368:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828975.702370:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2861
00000100:00080000:0.0:1787828975.702378:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828975.702381:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828975.702385:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828975.702394:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626893312/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787828975.702421:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828975.702464:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828975/real 1787828975]  req@ffff8cd0c8e9d2c0 x1874671626892928/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829000 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828975.702483:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d2c0 x1874671626892928/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829000 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828975.702505:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828975.702509:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828975.702514:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626892928:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828975.702523:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626893056:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828975.702557:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828975.702577:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828975/real 1787828975]  req@ffff8cd0c8e9e940 x1874671626893056/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829000 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828975.702591:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626893056/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829000 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828975.702601:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828975.702605:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828975.702609:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626893056:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828975.702634:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626893184:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787828975.702658:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828975.702670:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828975/real 1787828975]  req@ffff8cd0c8e9c3c0 x1874671626893184/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829000 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828975.702683:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626893184/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829000 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828975.702692:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828975.702695:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828975.702699:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626893184:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828975.702720:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626893312:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787828975.702743:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787828975.702766:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828975/real 1787828975]  req@ffff8cd0c8e9d680 x1874671626893312/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829000 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828975.702794:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626893312/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829000 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828975.702804:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828975.702807:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828975.702811:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626893312:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828975.894238:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828976.278240:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828976.918180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828977.302251:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828977.942178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828978.326228:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828978.966150:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828979.350256:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828979.990172:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787828980.374219:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828980.822150:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828980.822161:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828980.822166:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828980.822176:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2866
00000100:00080000:0.0:1787828980.822188:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828980.822195:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828980.822233:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626893440/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828980.822256:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828980.822259:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828980.822265:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828980.822267:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2866
00000100:00080000:0.0:1787828980.822270:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828980.822274:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828980.822282:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828980.822291:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626893568/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828980.822300:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:2.0:1787828980.822301:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626893440:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828980.822317:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828980.822319:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828980.822323:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2866
00000100:00080000:0.0:1787828980.822327:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828980.822330:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828980.822335:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828980.822351:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626893696/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828980.822359:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828980.822361:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828980.822363:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828980.822366:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2866
00000100:00080000:0.0:1787828980.822369:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828980.822371:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828980.822375:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828980.822384:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626893824/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787828980.822402:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626893568:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828980.822444:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626893696:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828980.822466:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626893824:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828981.014143:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828981.398268:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828982.038192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828982.422302:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828983.062201:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828983.446311:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:1.0:1787828983.894399:0:2345:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:1.0:1787828983.894449:0:2345:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1787828983.894467:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787828983.894507:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787828983.894514:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787828983.894524:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:2.0:1787828983.894595:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828980/real 1787828983]  req@ffff8cd0c8e9de00 x1874671626893568/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828983.894621:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626893568/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828983.894632:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828983.894636:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828983.894640:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626893568:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828983.894646:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828980/real 1787828983]  req@ffff8cd0c8e9cf00 x1874671626893696/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828983.894695:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626893696/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828983.894704:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828983.894708:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828983.894712:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626893696:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787828983.894719:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828980/real 1787828983]  req@ffff8cd0c8e9fc00 x1874671626893824/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828983.894732:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626893824/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828983.894741:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828983.894745:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828983.894748:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626893824:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787828983.894755:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828980/real 1787828983]  req@ffff8cd0c8e9ed00 x1874671626893440/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787828983.894768:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626893440/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787828983.894791:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787828983.894795:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787828983.894799:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626893440:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828984.086162:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828984.470292:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828985.110261:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828985.494312:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787828985.942201:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828985.942231:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787828985.942236:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828985.942247:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2871
00000100:00080000:2.0:1787828985.942258:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828985.942266:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787828985.942305:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626893952/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828985.942339:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828985.942342:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828985.942343:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828985.942346:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2871
00000100:00080000:2.0:1787828985.942349:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828985.942356:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828985.942367:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828985.942380:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626894080/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828985.942391:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828985.942395:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787828985.942397:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828985.942401:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2871
00000100:00080000:2.0:1787828985.942406:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828985.942411:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828985.942417:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828985.942429:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626893952:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787828985.942434:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626894208/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787828985.942467:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787828985.942471:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787828985.942473:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787828985.942478:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2871
00000100:00080000:2.0:1787828985.942498:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787828985.942504:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787828985.942511:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787828985.942530:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626894336/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828985.942531:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828985.942600:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828985/real 1787828985]  req@ffff8cd0c3dab840 x1874671626893952/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828985.942621:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626893952/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828985.942647:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828985.942652:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828985.942657:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626893952:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828985.942667:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626894080:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828985.942722:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828985.942737:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828985/real 1787828985]  req@ffff8cd0c3da8000 x1874671626894080/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828985.942750:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626894080/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828985.942758:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828985.942762:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828985.942765:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626894080:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828985.942791:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626894208:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828985.942816:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828985.942829:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828985/real 1787828985]  req@ffff8cd0c3dabc00 x1874671626894208/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828985.942842:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626894208/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828985.942850:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828985.942854:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828985.942857:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626894208:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828985.942865:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626894336:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828985.942886:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828985.942905:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828985/real 1787828985]  req@ffff8cd0c3daa580 x1874671626894336/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828985.942935:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626894336/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828985.942944:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828985.942948:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828985.942951:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626894336:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828986.134186:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828986.518215:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828987.158206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828987.542295:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828988.182148:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828988.566289:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828989.206219:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828989.590317:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828990.230210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828990.614291:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828991.062195:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828991.062207:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828991.062211:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828991.062221:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2876
00000100:00080000:0.0:1787828991.062232:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828991.062240:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828991.062277:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626894464/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828991.062320:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828991.062324:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828991.062327:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828991.062331:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2876
00000100:00080000:0.0:1787828991.062337:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828991.062343:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828991.062354:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828991.062355:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626894464:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787828991.062375:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626894592/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828991.062389:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828991.062393:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828991.062396:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828991.062400:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2876
00000100:00080000:0.0:1787828991.062405:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828991.062410:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828991.062416:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787828991.062429:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787828991.062431:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626894720/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828991.062444:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828991.062449:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828991.062451:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828991.062456:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2876
00000100:00080000:0.0:1787828991.062468:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828991.062474:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787828991.062480:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828991/real 1787828991]  req@ffff8cd0c8e9e580 x1874671626894464/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829016 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:0.0:1787828991.062489:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828991.062497:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626894464/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829016 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828991.062506:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:0.0:1787828991.062509:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626894848/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787828991.062512:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828991.062520:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626894464:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787828991.062547:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626894592:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828991.062574:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828991.062584:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828991/real 1787828991]  req@ffff8cd0c8e9ed00 x1874671626894592/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829016 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828991.062592:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626894592/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829016 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828991.062612:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828991.062615:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828991.062618:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626894592:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828991.062623:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626894720:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828991.062639:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828991.062647:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828991/real 1787828991]  req@ffff8cd0c8e9fc00 x1874671626894720/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829016 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828991.062655:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626894720/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829016 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828991.062661:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828991.062663:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828991.062665:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626894720:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828991.062668:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626894848:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828991.062678:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828991.062684:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828991/real 1787828991]  req@ffff8cd0c8e9cf00 x1874671626894848/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829016 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828991.062703:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626894848/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829016 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828991.062709:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828991.062711:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828991.062713:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626894848:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828991.254192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828991.638269:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828992.278178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828992.662249:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828993.302179:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828993.686213:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828994.326183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828994.710272:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828995.350236:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828995.734238:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787828996.182166:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828996.182176:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787828996.182181:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828996.182191:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2882
00000100:00080000:0.0:1787828996.182203:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828996.182210:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787828996.182247:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626894976/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828996.182278:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828996.182282:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787828996.182284:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828996.182288:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2882
00000100:00080000:0.0:1787828996.182294:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828996.182299:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828996.182309:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828996.182317:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626894976:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787828996.182327:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626895104/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787828996.182342:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828996.182359:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787828996.182362:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828996.182367:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2882
00000100:00080000:0.0:1787828996.182373:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787828996.182378:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:3.0:1787828996.182382:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:0.0:1787828996.182388:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787828996.182407:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626895232/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787828996.182428:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828996/real 1787828996]  req@ffff8cd0c8e9de00 x1874671626894976/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829021 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787828996.182438:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787828996.182443:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:3.0:1787828996.182447:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626894976/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829021 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787828996.182448:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787828996.182462:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2882
00000100:00080000:3.0:1787828996.182462:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828996.182469:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00080000:0.0:1787828996.182471:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1787828996.182473:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626894976:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787828996.182484:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787828996.182492:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787828996.182501:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626895104:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787828996.182509:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626895360/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787828996.182537:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828996.182550:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828996/real 1787828996]  req@ffff8cd0c8e9d680 x1874671626895104/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829021 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828996.182559:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626895104/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829021 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828996.182579:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828996.182582:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828996.182584:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626895104:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828996.182589:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626895232:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787828996.182606:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828996.182614:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828996/real 1787828996]  req@ffff8cd0c8e9c3c0 x1874671626895232/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829021 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828996.182621:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626895232/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829021 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828996.182627:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828996.182629:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828996.182631:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626895232:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787828996.182641:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626895360:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787828996.182652:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787828996.182659:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828996/real 1787828996]  req@ffff8cd0c8e9e940 x1874671626895360/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829021 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787828996.182677:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626895360/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829021 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787828996.182682:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787828996.182684:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787828996.182686:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626895360:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787828996.374158:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828996.758248:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828997.398212:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828997.782277:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828998.422155:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787828998.806243:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787828999.446144:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787828999.830251:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829000.470145:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829000.854235:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787829001.302179:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829001.302188:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787829001.302191:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829001.302199:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2887
00000100:00080000:0.0:1787829001.302209:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829001.302215:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829001.302246:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d2c0 x1874671626895488/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829001.302281:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829001.302284:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829001.302285:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829001.302288:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2887
00000100:00080000:0.0:1787829001.302290:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829001.302293:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829001.302300:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829001.302310:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626895616/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829001.302318:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829001.302321:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787829001.302322:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829001.302324:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2887
00000100:00080000:0.0:1787829001.302326:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829001.302329:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787829001.302333:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626895488:192.168.203.145@tcp:250:kworker.0
00000008:00100000:0.0:1787829001.302334:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829001.302344:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626895744/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829001.302352:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829001.302354:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829001.302356:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829001.302358:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2887
00000100:00080000:0.0:1787829001.302366:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829001.302370:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829001.302375:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829001.302389:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626895872/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787829001.302398:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829001.302450:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829001/real 1787829001]  req@ffff8cd0c8e9d2c0 x1874671626895488/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829026 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829001.302472:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d2c0 x1874671626895488/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829026 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829001.302484:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829001.302488:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829001.302492:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626895488:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829001.302501:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626895616:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829001.302555:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829001.302571:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829001/real 1787829001]  req@ffff8cd0c8e9f480 x1874671626895616/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829026 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829001.302585:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626895616/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829026 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829001.302594:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829001.302598:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829001.302601:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626895616:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829001.302627:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626895744:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829001.302649:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829001.302660:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829001/real 1787829001]  req@ffff8cd0c8e9f840 x1874671626895744/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829026 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829001.302673:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626895744/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829026 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829001.302682:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829001.302685:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829001.302688:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626895744:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829001.302694:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626895872:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829001.302710:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829001.302719:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829001/real 1787829001]  req@ffff8cd0c8e9c000 x1874671626895872/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829026 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829001.302745:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626895872/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829026 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829001.302752:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829001.302754:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829001.302756:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626895872:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829001.494184:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787829001.878261:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829002.518163:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829002.902269:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829003.542162:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787829003.926229:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829004.566174:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829004.950248:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829005.590162:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787829005.974221:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787829006.422161:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829006.422168:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787829006.422172:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829006.422180:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2892
00000100:00080000:0.0:1787829006.422189:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829006.422195:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829006.422229:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626896000/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829006.422255:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829006.422259:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829006.422261:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829006.422265:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2892
00000100:00080000:0.0:1787829006.422270:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829006.422276:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829006.422286:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829006.422290:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626896000:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829006.422307:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626896128/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829006.422321:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829006.422338:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787829006.422339:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829006.422343:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2892
00000800:00000100:3.0:1787829006.422347:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787829006.422350:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829006.422356:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829006.422362:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829006.422374:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626896256/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829006.422381:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829006.422384:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829006.422386:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829006.422389:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2892
00000100:00080000:0.0:1787829006.422391:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1787829006.422394:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829006/real 1787829006]  req@ffff8cd0c8e9cb40 x1874671626896000/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829031 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829006.422397:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829006.422404:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829006.422414:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626896384/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829006.422430:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626896000/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829031 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829006.422440:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829006.422443:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829006.422447:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626896000:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829006.422467:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626896128:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829006.422507:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829006.422519:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829006/real 1787829006]  req@ffff8cd0c8e9c000 x1874671626896128/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829031 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829006.422528:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626896128/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829031 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829006.422533:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829006.422536:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829006.422538:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626896128:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829006.422542:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626896256:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829006.422556:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829006.422563:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829006/real 1787829006]  req@ffff8cd0c8e9f840 x1874671626896256/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829031 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829006.422570:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626896256/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829031 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829006.422575:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829006.422577:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829006.422579:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626896256:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829006.422582:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626896384:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829006.422592:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829006.422598:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829006/real 1787829006]  req@ffff8cd0c8e9f480 x1874671626896384/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829031 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829006.422618:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626896384/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829031 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829006.422624:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829006.422626:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829006.422628:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626896384:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829006.614181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829006.998231:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829007.638141:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787829008.022267:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829008.662181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829009.046264:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829009.686191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787829010.070223:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829010.710198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829011.094256:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787829011.542197:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829011.542212:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787829011.542218:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829011.542231:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2897
00000100:00080000:0.0:1787829011.542245:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829011.542253:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829011.542299:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d2c0 x1874671626896512/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829011.542347:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829011.542350:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829011.542352:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829011.542355:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2897
00000100:00080000:0.0:1787829011.542358:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829011.542361:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829011.542372:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829011.542383:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626896640/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829011.542392:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787829011.542395:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626896512:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787829011.542396:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787829011.542399:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829011.542406:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2897
00000100:00080000:0.0:1787829011.542409:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829011.542411:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829011.542415:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829011.542425:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626896768/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829011.542433:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829011.542436:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829011.542437:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829011.542439:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2897
00000100:00080000:0.0:1787829011.542449:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829011.542453:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829011.542473:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829011.542485:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626896896/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787829011.542493:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829011.542558:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829011/real 1787829011]  req@ffff8cd0c8e9d2c0 x1874671626896512/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829036 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829011.542579:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d2c0 x1874671626896512/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829036 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829011.542592:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829011.542597:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829011.542602:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626896512:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829011.542630:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626896640:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829011.542658:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829011.542673:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829011/real 1787829011]  req@ffff8cd0c8e9e940 x1874671626896640/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829036 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829011.542687:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626896640/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829036 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829011.542698:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829011.542701:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829011.542705:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626896640:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829011.542713:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626896768:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829011.542732:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829011.542740:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829011/real 1787829011]  req@ffff8cd0c8e9c3c0 x1874671626896768/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829036 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829011.542748:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626896768/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829036 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829011.542754:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829011.542757:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829011.542759:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626896768:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829011.542764:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626896896:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829011.542776:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829011.542781:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829011/real 1787829011]  req@ffff8cd0c8e9d680 x1874671626896896/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829036 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829011.542810:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626896896/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829036 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829011.542819:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829011.542823:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829011.542826:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626896896:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829011.734167:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787829012.118329:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829012.758240:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829013.142289:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829013.782181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787829014.166300:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829014.806247:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829015.190236:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829015.830188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787829016.214320:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787829016.662197:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829016.662210:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787829016.662216:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829016.662229:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2902
00000100:00080000:0.0:1787829016.662243:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829016.662252:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829016.662301:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626897024/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829016.662352:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829016.662357:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829016.662361:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829016.662366:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2902
00000100:00080000:0.0:1787829016.662371:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829016.662376:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829016.662390:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829016.662407:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626897152/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829016.662436:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626897024:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787829016.662437:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829016.662471:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787829016.662474:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829016.662480:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2902
00000100:00080000:0.0:1787829016.662485:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829016.662490:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829016.662497:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829016.662511:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626897280/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829016.662523:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829016.662528:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829016.662531:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:3.0:1787829016.662532:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787829016.662540:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2902
00000100:00080000:0.0:1787829016.662545:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829016.662551:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829016.662557:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829016.662569:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626897408/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829016.662589:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829016/real 1787829016]  req@ffff8cd0c8e9de00 x1874671626897024/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829041 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829016.662613:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626897024/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829041 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829016.662622:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829016.662625:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829016.662628:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626897024:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829016.662652:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626897152:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829016.662680:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829016.662690:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829016/real 1787829016]  req@ffff8cd0c8e9cf00 x1874671626897152/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829041 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829016.662699:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626897152/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829041 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829016.662705:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829016.662707:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829016.662709:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626897152:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829016.662713:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626897280:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829016.662725:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829016.662730:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829016/real 1787829016]  req@ffff8cd0c8e9fc00 x1874671626897280/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829041 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829016.662738:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626897280/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829041 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829016.662743:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829016.662745:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829016.662747:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626897280:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829016.662750:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626897408:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829016.662759:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829016.662765:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829016/real 1787829016]  req@ffff8cd0c8e9ed00 x1874671626897408/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829041 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829016.662782:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626897408/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829041 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829016.662788:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829016.662790:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829016.662792:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626897408:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829016.854244:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829017.238326:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829017.878175:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787829018.262283:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829018.902227:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829019.286365:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829019.926157:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787829020.310232:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829020.950144:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829021.334220:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829021.782193:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829021.782202:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829021.782206:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829021.782213:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2907
00000100:00080000:1.0:1787829021.782223:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829021.782228:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829021.782261:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626897536/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829021.782281:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829021.782284:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829021.782285:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829021.782288:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2907
00000100:00080000:1.0:1787829021.782290:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829021.782292:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829021.782300:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829021.782308:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626897664/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829021.782328:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626897536:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787829021.782332:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829021.782337:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829021.782339:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829021.782342:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2907
00000100:00080000:1.0:1787829021.782345:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829021.782347:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829021.782351:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829021.782360:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626897792/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829021.782367:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829021.782370:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829021.782371:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829021.782373:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2907
00000100:00080000:1.0:1787829021.782375:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829021.782377:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829021.782380:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829021.782400:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626897920/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787829021.782421:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829021.782463:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829021/real 1787829021]  req@ffff8cd0d48f4b40 x1874671626897536/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829046 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829021.782482:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626897536/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829046 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829021.782493:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829021.782497:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829021.782502:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626897536:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829021.782526:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626897664:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829021.782557:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829021.782572:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829021/real 1787829021]  req@ffff8cd0d48f6580 x1874671626897664/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829046 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829021.782585:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626897664/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829046 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829021.782594:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829021.782599:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829021.782602:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626897664:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829021.782608:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626897792:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829021.782630:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829021.782642:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829021/real 1787829021]  req@ffff8cd0d48f7480 x1874671626897792/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829046 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829021.782654:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626897792/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829046 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829021.782662:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829021.782666:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829021.782669:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626897792:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829021.782676:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626897920:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829021.782694:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829021.782704:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829021/real 1787829021]  req@ffff8cd0d48f4000 x1874671626897920/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829046 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829021.782729:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626897920/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829046 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829021.782737:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829021.782740:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829021.782744:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626897920:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829021.974176:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787829022.358254:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829022.998169:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829023.382255:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829024.022209:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787829024.406257:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829025.046146:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829025.430190:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829026.070185:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787829026.454210:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829026.902148:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829026.902156:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829026.902174:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829026.902182:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2912
00000100:00080000:1.0:1787829026.902192:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829026.902198:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829026.902229:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626898048/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829026.902257:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829026.902260:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829026.902261:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829026.902263:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2912
00000100:00080000:1.0:1787829026.902266:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829026.902268:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829026.902277:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829026.902286:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626898176/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829026.902294:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787829026.902295:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626898048:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787829026.902300:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829026.902304:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829026.902312:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2912
00000100:00080000:1.0:1787829026.902314:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829026.902317:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829026.902320:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829026.902330:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626898304/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829026.902347:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829026.902364:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829026.902366:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829026.902373:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2912
00000800:00000100:3.0:1787829026.902373:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787829026.902380:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829026.902384:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829026.902389:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829026.902405:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626898432/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829026.902418:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829026/real 1787829026]  req@ffff8cd0d48f4000 x1874671626898048/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829051 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829026.902433:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626898048/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829051 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829026.902442:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829026.902445:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829026.902448:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626898048:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829026.902466:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626898176:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829026.902492:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829026.902500:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829026/real 1787829026]  req@ffff8cd0d48f7480 x1874671626898176/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829051 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829026.902509:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626898176/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829051 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829026.902515:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829026.902517:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829026.902519:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626898176:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829026.902523:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626898304:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829026.902533:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829026.902539:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829026/real 1787829026]  req@ffff8cd0d48f6580 x1874671626898304/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829051 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829026.902557:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626898304/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829051 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829026.902565:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829026.902569:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829026.902572:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626898304:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829026.902578:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626898432:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829026.902599:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829026.902610:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829026/real 1787829026]  req@ffff8cd0d48f4b40 x1874671626898432/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829051 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829026.902632:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626898432/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829051 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829026.902638:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829026.902640:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829026.902642:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626898432:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829027.094151:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829027.478193:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829028.118194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787829028.502221:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829029.142147:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829029.526206:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829030.166226:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787829030.550225:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829031.190162:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829031.574313:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829032.022256:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829032.022269:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829032.022274:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829032.022284:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2917
00000100:00080000:2.0:1787829032.022295:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829032.022303:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829032.022354:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626898560/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829032.022384:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829032.022388:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829032.022390:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829032.022395:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2917
00000100:00080000:2.0:1787829032.022399:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829032.022403:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829032.022430:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829032.022434:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626898560:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787829032.022448:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626898688/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829032.022462:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829032.022466:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829032.022469:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829032.022473:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2917
00000100:00080000:2.0:1787829032.022487:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829032.022493:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829032.022499:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787829032.022500:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787829032.022516:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626898816/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829032.022527:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829032.022531:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829032.022533:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829032.022538:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2917
00000100:00080000:2.0:1787829032.022542:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1787829032.022547:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829032/real 1787829032]  req@ffff8cd0c3da9e00 x1874671626898560/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829057 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829032.022549:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829032.022559:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829032.022577:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daad00 x1874671626898944/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829032.022581:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626898560/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829057 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829032.022605:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829032.022611:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829032.022615:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626898560:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829032.022642:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626898688:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829032.022673:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829032.022686:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829032/real 1787829032]  req@ffff8cd0c3da92c0 x1874671626898688/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829057 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829032.022700:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626898688/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829057 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829032.022715:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829032.022719:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829032.022723:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626898688:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829032.022730:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626898816:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829032.022750:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829032.022761:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829032/real 1787829032]  req@ffff8cd0c3daa1c0 x1874671626898816/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829057 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829032.022774:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626898816/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829057 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829032.022783:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829032.022786:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829032.022790:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626898816:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829032.022796:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626898944:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829032.022814:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829032.022824:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829032/real 1787829032]  req@ffff8cd0c3daad00 x1874671626898944/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829057 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829032.022851:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daad00 x1874671626898944/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829057 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829032.022860:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829032.022863:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829032.022867:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626898944:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829032.214142:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787829032.598233:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829033.238127:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829033.622235:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829034.262148:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787829034.646214:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829035.286140:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829035.670372:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829036.310153:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787829036.694243:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787829037.142194:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829037.142201:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787829037.142205:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829037.142213:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2922
00000100:00080000:0.0:1787829037.142223:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829037.142228:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829037.142261:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626899072/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829037.142311:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829037.142314:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829037.142315:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829037.142318:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2922
00000100:00080000:0.0:1787829037.142321:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829037.142324:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829037.142332:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829037.142344:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626899200/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829037.142354:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829037.142356:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787829037.142360:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1787829037.142362:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626899072:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787829037.142363:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2922
00000100:00080000:0.0:1787829037.142368:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829037.142373:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829037.142376:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829037.142383:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626899328/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829037.142390:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829037.142393:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829037.142394:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829037.142396:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2922
00000100:00080000:0.0:1787829037.142405:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829037.142408:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829037.142412:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829037.142426:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626899456/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787829037.142445:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829037.142496:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829037/real 1787829037]  req@ffff8cd0c8e9e580 x1874671626899072/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829062 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829037.142518:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626899072/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829062 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829037.142529:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829037.142534:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829037.142539:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626899072:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829037.142563:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626899200:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829037.142600:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829037.142616:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829037/real 1787829037]  req@ffff8cd0c8e9ed00 x1874671626899200/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829062 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829037.142630:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626899200/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829062 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829037.142639:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829037.142643:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829037.142647:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626899200:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829037.142654:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626899328:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829037.142675:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829037.142687:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829037/real 1787829037]  req@ffff8cd0c8e9fc00 x1874671626899328/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829062 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829037.142701:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626899328/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829062 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829037.142711:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829037.142716:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829037.142720:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626899328:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829037.142727:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626899456:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829037.142748:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829037.142762:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829037/real 1787829037]  req@ffff8cd0c8e9cf00 x1874671626899456/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829062 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829037.142793:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626899456/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829062 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829037.142802:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829037.142805:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829037.142808:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626899456:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829037.334134:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829037.718228:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829038.358219:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787829038.742299:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829039.382157:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829039.766316:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829040.406191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:1.0:1787829040.790217:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829041.430134:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829041.814269:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829042.262202:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829042.262210:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829042.262213:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829042.262221:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2928
00000100:00080000:1.0:1787829042.262231:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829042.262236:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829042.262285:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626899584/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829042.262306:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829042.262322:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829042.262324:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829042.262328:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2928
00000100:00080000:1.0:1787829042.262331:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829042.262335:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829042.262344:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829042.262348:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626899584:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829042.262356:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626899712/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829042.262367:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829042.262369:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829042.262370:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829042.262373:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2928
00000100:00080000:1.0:1787829042.262376:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829042.262378:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829042.262381:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829042.262388:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626899840/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829042.262395:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829042.262398:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829042.262399:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829042.262402:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2928
00000800:00000100:3.0:1787829042.262406:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787829042.262407:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829042.262413:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829042.262422:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829042.262429:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626899968/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829042.262453:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829042/real 1787829042]  req@ffff8cd0d48f7840 x1874671626899584/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829067 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829042.262468:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626899584/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829067 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829042.262476:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829042.262479:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829042.262483:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626899584:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829042.262500:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626899712:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829042.262528:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829042.262541:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829042/real 1787829042]  req@ffff8cd0d48f5a40 x1874671626899712/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829067 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829042.262555:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626899712/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829067 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829042.262565:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829042.262569:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829042.262572:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626899712:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829042.262577:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626899840:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829042.262602:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829042.262610:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829042/real 1787829042]  req@ffff8cd0d48f43c0 x1874671626899840/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829067 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829042.262618:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626899840/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829067 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829042.262624:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829042.262626:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829042.262628:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626899840:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829042.262632:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626899968:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829042.262649:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829042.262656:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829042/real 1787829042]  req@ffff8cd0d48f70c0 x1874671626899968/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829067 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829042.262677:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626899968/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829067 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829042.262685:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829042.262688:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829042.262692:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626899968:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829042.454154:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829042.838263:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829043.478188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829043.862275:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829044.502151:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829044.886183:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829045.526182:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829045.910271:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829046.550170:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829046.934202:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829047.382171:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829047.382181:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829047.382185:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829047.382196:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2933
00000100:00080000:1.0:1787829047.382221:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829047.382229:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829047.382267:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626900096/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829047.382295:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829047.382299:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829047.382301:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829047.382305:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2933
00000100:00080000:1.0:1787829047.382310:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829047.382316:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829047.382328:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829047.382330:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626900096:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829047.382346:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626900224/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829047.382361:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829047.382364:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829047.382367:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829047.382371:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2933
00000100:00080000:1.0:1787829047.382402:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829047.382408:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829047.382415:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829047.382427:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626900224:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787829047.382434:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626900352/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829047.382448:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829047.382453:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:3.0:1787829047.382456:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626900352:192.168.203.145@tcp:8:kworker.0
00000100:00080000:1.0:1787829047.382457:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829047.382466:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2933
00000100:00080000:1.0:1787829047.382473:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829047.382478:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829047.382483:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829047.382498:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626900480/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829047.382554:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626900480:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829047.574172:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829047.958273:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829048.598146:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829048.982166:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829049.622189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829050.006271:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:2.0:1787829050.454346:0:2344:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:2.0:1787829050.454382:0:2344:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:2.0:1787829050.454406:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787829050.454436:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787829050.454443:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787829050.454451:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:3.0:1787829050.454490:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829047/real 1787829050]  req@ffff8cd0d48f61c0 x1874671626900480/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829072 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829050.454530:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626900480/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829072 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829050.454542:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829050.454547:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829050.454552:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626900480:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829050.454560:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829047/real 1787829050]  req@ffff8cd0d48f6d00 x1874671626900352/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829072 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829050.454573:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626900352/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829072 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829050.454582:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829050.454586:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829050.454589:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626900352:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829050.454596:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829047/real 1787829050]  req@ffff8cd0d48f52c0 x1874671626900224/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829072 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829050.454611:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626900224/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829072 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829050.454620:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829050.454624:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829050.454627:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626900224:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829050.454634:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829047/real 1787829050]  req@ffff8cd0d48f7c00 x1874671626900096/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829072 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829050.454656:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626900096/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829072 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829050.454666:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829050.454669:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829050.454673:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626900096:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829050.646167:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829051.030221:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829051.670195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829052.054228:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829052.502207:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829052.502215:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829052.502218:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829052.502226:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2938
00000100:00080000:1.0:1787829052.502235:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829052.502240:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829052.502278:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626900608/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829052.502310:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829052.502334:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829052.502336:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829052.502340:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2938
00000100:00080000:1.0:1787829052.502344:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829052.502349:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829052.502359:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829052.502360:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626900608:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829052.502371:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626900736/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829052.502382:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829052.502385:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829052.502386:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829052.502389:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2938
00000100:00080000:1.0:1787829052.502392:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829052.502396:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829052.502400:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829052.502408:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626900864/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829052.502416:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:3.0:1787829052.502419:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787829052.502420:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829052.502421:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829052.502425:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2938
00000100:00080000:1.0:1787829052.502428:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829052.502431:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829052.502441:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829052.502451:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626900992/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829052.502456:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829052/real 1787829052]  req@ffff8cd0d48f7c00 x1874671626900608/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829077 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829052.502473:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626900608/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829077 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829052.502483:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829052.502488:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829052.502492:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626900608:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829052.502514:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626900736:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829052.502534:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829052.502543:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829052/real 1787829052]  req@ffff8cd0d48f52c0 x1874671626900736/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829077 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829052.502551:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626900736/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829077 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829052.502567:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829052.502570:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829052.502574:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626900736:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829052.502580:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626900864:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829052.502603:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829052.502615:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829052/real 1787829052]  req@ffff8cd0d48f6d00 x1874671626900864/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829077 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829052.502626:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626900864/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829077 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829052.502634:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829052.502637:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829052.502639:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626900864:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829052.502646:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626900992:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829052.502664:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829052.502682:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829052/real 1787829052]  req@ffff8cd0d48f61c0 x1874671626900992/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829077 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829052.502694:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626900992/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829077 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829052.502702:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829052.502706:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829052.502715:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626900992:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829052.694230:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829053.078304:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829053.718264:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829054.102282:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829054.742203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829055.126283:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829055.766204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829056.150201:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829056.790166:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829057.174214:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787829057.622202:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829057.622210:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787829057.622214:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829057.622223:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2943
00000100:00080000:0.0:1787829057.622233:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829057.622238:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829057.622269:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626901120/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829057.622288:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829057.622291:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829057.622292:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829057.622295:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2943
00000100:00080000:0.0:1787829057.622301:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829057.622317:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829057.622325:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829057.622336:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626901248/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829057.622346:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626901120:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787829057.622347:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829057.622355:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787829057.622358:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829057.622361:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2943
00000100:00080000:0.0:1787829057.622364:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829057.622367:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829057.622371:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829057.622379:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626901376/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829057.622386:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829057.622389:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829057.622390:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829057.622393:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2943
00000100:00080000:0.0:1787829057.622395:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829057.622397:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829057.622401:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829057.622407:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626901504/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787829057.622418:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829057.622458:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829057/real 1787829057]  req@ffff8cd0c8e9de00 x1874671626901120/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829082 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829057.622477:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626901120/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829082 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829057.622497:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829057.622502:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829057.622506:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626901120:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829057.622537:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626901248:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829057.622564:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829057.622579:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829057/real 1787829057]  req@ffff8cd0c8e9d680 x1874671626901248/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829082 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829057.622602:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626901248/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829082 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829057.622611:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829057.622621:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829057.622625:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626901248:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829057.622632:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626901376:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829057.622665:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829057.622678:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829057/real 1787829057]  req@ffff8cd0c8e9c3c0 x1874671626901376/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829082 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829057.622701:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626901376/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829082 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829057.622710:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829057.622714:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829057.622717:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626901376:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829057.622724:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626901504:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829057.622745:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829057.622757:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829057/real 1787829057]  req@ffff8cd0c8e9e940 x1874671626901504/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829082 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829057.622769:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626901504/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829082 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829057.622778:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829057.622782:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829057.622785:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626901504:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829057.814188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829058.198262:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829058.838169:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829059.222283:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829059.862195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829060.246279:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829060.886153:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829061.270197:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829061.910193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829062.294335:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787829062.742221:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829062.742236:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787829062.742242:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829062.742255:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2948
00000100:00080000:0.0:1787829062.742271:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829062.742282:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829062.742342:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d2c0 x1874671626901632/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829062.742388:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829062.742397:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829062.742399:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829062.742405:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2948
00000100:00080000:0.0:1787829062.742410:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829062.742416:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829062.742440:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829062.742444:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626901632:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829062.742466:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626901760/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829062.742482:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829062.742486:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787829062.742488:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829062.742493:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2948
00000100:00080000:0.0:1787829062.742498:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829062.742502:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:3.0:1787829062.742521:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:0.0:1787829062.742537:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829062.742557:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626901888/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829062.742569:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829062.742574:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829062.742576:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829062.742583:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2948
00000100:00100000:3.0:1787829062.742584:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829062/real 1787829062]  req@ffff8cd0c8e9d2c0 x1874671626901632/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829087 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829062.742589:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829062.742599:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829062.742610:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829062.742620:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d2c0 x1874671626901632/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829087 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829062.742632:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:0.0:1787829062.742636:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626902016/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829062.742638:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829062.742647:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626901632:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829062.742687:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626901760:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829062.742713:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829062.742734:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829062/real 1787829062]  req@ffff8cd0c8e9f480 x1874671626901760/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829087 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829062.742744:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626901760/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829087 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829062.742750:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829062.742753:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829062.742761:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626901760:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829062.742766:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626901888:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829062.742786:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829062.742799:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829062/real 1787829062]  req@ffff8cd0c8e9f840 x1874671626901888/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829087 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829062.742812:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626901888/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829087 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829062.742835:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829062.742838:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829062.742842:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626901888:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829062.742849:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626902016:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829062.742885:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829062.742898:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829062/real 1787829062]  req@ffff8cd0c8e9c000 x1874671626902016/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829087 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829062.742908:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626902016/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829087 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829062.742914:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829062.742916:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829062.742918:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626902016:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829062.934280:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829063.318245:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829063.958250:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829064.342285:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829064.982198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829065.366244:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829066.006171:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829066.390256:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829067.030168:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829067.414179:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829067.862212:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829067.862222:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829067.862226:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829067.862235:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2953
00000100:00080000:2.0:1787829067.862246:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829067.862253:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829067.862291:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daad00 x1874671626902144/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829067.862316:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829067.862319:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829067.862321:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829067.862324:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2953
00000100:00080000:2.0:1787829067.862327:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829067.862331:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829067.862343:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829067.862377:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626902272/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829067.862392:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787829067.862394:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626902144:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787829067.862412:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829067.862427:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829067.862431:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2953
00000100:00080000:2.0:1787829067.862436:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829067.862440:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829067.862445:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829067.862456:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626902400/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829067.862464:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829067.862468:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829067.862471:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:3.0:1787829067.862472:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787829067.862492:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2953
00000100:00080000:2.0:1787829067.862496:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829067.862499:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829067.862503:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829067.862511:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626902528/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829067.862536:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829067/real 1787829067]  req@ffff8cd0c3daad00 x1874671626902144/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829067.862563:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daad00 x1874671626902144/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829067.862571:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829067.862573:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829067.862576:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626902144:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829067.862595:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626902272:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829067.862612:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829067.862631:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829067/real 1787829067]  req@ffff8cd0c3daa1c0 x1874671626902272/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829067.862638:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626902272/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829067.862644:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829067.862646:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829067.862648:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626902272:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829067.862652:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626902400:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829067.862674:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829067.862681:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829067/real 1787829067]  req@ffff8cd0c3da92c0 x1874671626902400/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829067.862689:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626902400/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829067.862694:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829067.862696:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829067.862698:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626902400:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829067.862702:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626902528:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829067.862712:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829067.862718:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829067/real 1787829067]  req@ffff8cd0c3da9e00 x1874671626902528/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829067.862725:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626902528/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829067.862730:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829067.862731:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829067.862733:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626902528:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829068.054154:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829068.438224:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829069.078131:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829069.462231:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787829070.102217:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829070.486229:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787829071.126191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829071.510215:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787829072.150183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829072.534252:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829072.982203:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829072.982213:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829072.982235:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829072.982244:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2958
00000100:00080000:2.0:1787829072.982256:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829072.982263:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829072.982311:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626902656/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829072.982339:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829072.982343:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829072.982345:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829072.982350:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2958
00000100:00080000:2.0:1787829072.982355:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829072.982360:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829072.982372:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829072.982383:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626902656:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787829072.982389:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626902784/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829072.982404:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829072.982409:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829072.982411:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829072.982416:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2958
00000100:00080000:2.0:1787829072.982430:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829072.982435:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:3.0:1787829072.982438:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:2.0:1787829072.982444:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829072.982471:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626902912/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829072.982495:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829072/real 1787829072]  req@ffff8cd0c3daa580 x1874671626902656/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829097 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829072.982497:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829072.982502:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829072.982505:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829072.982516:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2958
00000100:00100000:3.0:1787829072.982525:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626902656/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829097 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829072.982529:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829072.982537:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787829072.982541:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00080000:3.0:1787829072.982544:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000008:00100000:2.0:1787829072.982551:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829072.982551:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626902656:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787829072.982572:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626903040/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829072.982585:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626902784:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829072.982617:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829072.982632:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829072/real 1787829072]  req@ffff8cd0c3dabc00 x1874671626902784/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829097 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829072.982647:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626902784/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829097 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829072.982656:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829072.982660:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829072.982664:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626902784:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829072.982670:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626902912:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829072.982693:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829072.982707:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829072/real 1787829072]  req@ffff8cd0c3da8000 x1874671626902912/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829097 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829072.982719:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626902912/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829097 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829072.982753:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829072.982757:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829072.982761:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626902912:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829072.982792:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626903040:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829072.982817:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829072.982832:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829072/real 1787829072]  req@ffff8cd0c3dab840 x1874671626903040/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829097 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829072.982844:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626903040/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829097 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829072.982873:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829072.982877:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829072.982882:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626903040:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787829073.174179:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829073.558201:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829074.198221:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829074.582231:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829075.222188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829075.606179:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829076.246173:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829076.630214:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829077.270140:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829077.654186:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829078.102202:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829078.102211:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829078.102215:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829078.102223:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2963
00000100:00080000:1.0:1787829078.102233:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829078.102239:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829078.102275:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626903168/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829078.102304:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829078.102308:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829078.102311:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829078.102315:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2963
00000100:00080000:1.0:1787829078.102319:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829078.102323:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829078.102348:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829078.102360:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626903296/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829078.102370:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787829078.102372:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626903168:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787829078.102373:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829078.102377:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829078.102383:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2963
00000100:00080000:1.0:1787829078.102422:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829078.102425:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829078.102431:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829078.102441:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626903424/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829078.102450:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:3.0:1787829078.102450:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787829078.102456:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829078.102458:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829078.102462:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2963
00000100:00080000:1.0:1787829078.102465:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829078.102469:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829078.102472:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829078.102482:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626903552/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829078.102510:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829078/real 1787829078]  req@ffff8cd0d48f70c0 x1874671626903168/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829103 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829078.102528:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626903168/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829103 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829078.102556:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829078.102578:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829078.102583:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626903168:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829078.102616:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626903296:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829078.102639:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829078.102650:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829078/real 1787829078]  req@ffff8cd0d48f43c0 x1874671626903296/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829103 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829078.102659:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626903296/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829103 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829078.102665:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829078.102668:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829078.102672:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626903296:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829078.102679:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626903424:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829078.102702:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829078.102724:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829078/real 1787829078]  req@ffff8cd0d48f5a40 x1874671626903424/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829103 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829078.102738:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626903424/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829103 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829078.102744:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829078.102747:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829078.102749:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626903424:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829078.102760:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626903552:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829078.102777:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829078.102786:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829078/real 1787829078]  req@ffff8cd0d48f7840 x1874671626903552/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829103 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829078.102801:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626903552/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829103 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829078.102807:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829078.102809:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829078.102812:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626903552:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829078.294136:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829078.678313:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829079.318247:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829079.702199:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829080.342212:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829080.726281:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829081.366223:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829081.750217:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829082.390138:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829082.774238:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829083.222187:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829083.222195:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829083.222199:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829083.222207:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2969
00000100:00080000:2.0:1787829083.222216:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829083.222222:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829083.222253:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626903680/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829083.222273:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829083.222286:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829083.222287:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829083.222291:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2969
00000100:00080000:2.0:1787829083.222295:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829083.222299:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829083.222307:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829083.222319:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626903808/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829083.222320:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626903680:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787829083.222335:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829083.222337:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829083.222339:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829083.222342:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2969
00000100:00080000:2.0:1787829083.222344:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829083.222346:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829083.222350:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829083.222359:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626903936/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829083.222366:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829083.222368:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829083.222370:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829083.222372:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2969
00000100:00080000:2.0:1787829083.222375:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829083.222378:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829083.222386:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787829083.222392:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787829083.222395:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626904064/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829083.222443:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829083/real 1787829083]  req@ffff8cd0c3dab0c0 x1874671626903680/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829108 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829083.222465:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626903680/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829108 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829083.222476:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829083.222481:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829083.222485:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626903680:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829083.222518:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626903808:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829083.222546:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829083.222561:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829083/real 1787829083]  req@ffff8cd0c3da9a40 x1874671626903808/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829108 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829083.222574:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626903808/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829108 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829083.222583:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829083.222586:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829083.222590:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626903808:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829083.222597:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626903936:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829083.222618:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829083.222630:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829083/real 1787829083]  req@ffff8cd0c3da8780 x1874671626903936/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829108 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829083.222643:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626903936/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829108 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829083.222651:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829083.222654:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829083.222658:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626903936:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829083.222664:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626904064:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829083.222684:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829083.222696:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829083/real 1787829083]  req@ffff8cd0c3daa940 x1874671626904064/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829108 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829083.222716:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626904064/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829108 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829083.222724:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829083.222728:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829083.222731:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626904064:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829083.414176:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829083.798230:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829084.438248:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829084.822231:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829085.462150:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829085.846156:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829086.486149:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829086.870225:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829087.510200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829087.894229:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787829088.342181:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829088.342190:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787829088.342207:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829088.342215:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2974
00000100:00080000:0.0:1787829088.342225:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829088.342230:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829088.342282:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626904192/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829088.342316:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829088.342320:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829088.342322:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829088.342332:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2974
00000100:00080000:0.0:1787829088.342337:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829088.342341:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829088.342353:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829088.342368:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626904320/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829088.342380:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626904192:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787829088.342392:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829088.342400:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787829088.342404:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829088.342428:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2974
00000100:00080000:0.0:1787829088.342435:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829088.342441:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829088.342451:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829088.342472:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626904448/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787829088.342478:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787829088.342518:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829088.342524:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829088.342527:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829088.342552:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2974
00000100:00100000:3.0:1787829088.342553:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829088/real 1787829088]  req@ffff8cd0c8e9cb40 x1874671626904192/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829113 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829088.342562:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829088.342572:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787829088.342576:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626904192/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829113 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:0.0:1787829088.342583:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00080000:3.0:1787829088.342587:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829088.342592:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829088.342596:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626904192:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829088.342605:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626904576/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829088.342621:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626904320:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829088.342644:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829088.342656:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829088/real 1787829088]  req@ffff8cd0c8e9c000 x1874671626904320/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829113 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829088.342674:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626904320/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829113 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829088.342680:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829088.342683:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829088.342685:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626904320:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829088.342690:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626904448:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829088.342705:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829088.342712:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829088/real 1787829088]  req@ffff8cd0c8e9f840 x1874671626904448/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829113 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829088.342721:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626904448/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829113 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829088.342727:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829088.342728:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829088.342731:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626904448:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829088.342741:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626904576:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829088.342753:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829088.342760:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829088/real 1787829088]  req@ffff8cd0c8e9f480 x1874671626904576/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829113 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829088.342768:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626904576/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829113 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829088.342783:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829088.342786:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829088.342788:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626904576:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829088.534236:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829088.918223:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829089.558196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829089.942222:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829090.582199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829090.966309:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829091.606188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829091.990221:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829092.630165:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829093.014236:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829093.462196:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829093.462204:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829093.462207:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829093.462216:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2979
00000100:00080000:2.0:1787829093.462226:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829093.462231:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829093.462265:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626904704/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829093.462289:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829093.462292:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829093.462294:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829093.462301:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2979
00000100:00080000:2.0:1787829093.462303:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829093.462306:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829093.462314:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829093.462323:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8b40 x1874671626904832/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829093.462339:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626904704:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787829093.462378:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829093.462381:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829093.462382:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829093.462386:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2979
00000100:00080000:2.0:1787829093.462390:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829093.462392:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829093.462397:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829093.462407:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626904960/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829093.462419:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829093.462425:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000800:00000100:3.0:1787829093.462426:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787829093.462427:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829093.462434:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2979
00000100:00080000:2.0:1787829093.462438:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829093.462441:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829093.462445:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829093.462453:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626905088/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829093.462480:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829093/real 1787829093]  req@ffff8cd0c3da8f00 x1874671626904704/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829118 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829093.462495:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626904704/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829118 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829093.462504:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829093.462507:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829093.462511:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626904704:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829093.462530:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626904832:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829093.462548:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829093.462558:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829093/real 1787829093]  req@ffff8cd0c3da8b40 x1874671626904832/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829118 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829093.462572:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8b40 x1874671626904832/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829118 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829093.462589:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829093.462594:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829093.462598:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626904832:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829093.462605:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626904960:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829093.462631:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829093.462645:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829093/real 1787829093]  req@ffff8cd0c3da83c0 x1874671626904960/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829118 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829093.462658:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626904960/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829118 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829093.462667:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829093.462670:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829093.462673:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626904960:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829093.462678:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626905088:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829093.462693:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829093.462701:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829093/real 1787829093]  req@ffff8cd0c3da9680 x1874671626905088/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829118 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829093.462708:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626905088/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829118 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829093.462713:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829093.462715:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829093.462723:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626905088:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829093.654179:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829094.038209:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829094.678178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829095.062192:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829095.702169:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829096.086174:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829096.726226:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829097.110270:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829097.750164:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829098.134283:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829098.582208:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829098.582219:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829098.582240:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829098.582249:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2984
00000100:00080000:2.0:1787829098.582260:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829098.582268:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829098.582304:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626905216/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829098.582338:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829098.582342:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829098.582344:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829098.582347:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2984
00000100:00080000:2.0:1787829098.582351:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829098.582356:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829098.582367:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829098.582383:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626905216:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787829098.582384:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626905344/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829098.582403:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829098.582407:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829098.582409:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829098.582413:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2984
00000100:00080000:2.0:1787829098.582417:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829098.582420:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829098.582425:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829098.582435:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8b40 x1874671626905472/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787829098.582455:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787829098.582459:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829098.582464:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829098.582466:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829098.582473:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2984
00000100:00080000:2.0:1787829098.582478:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829098.582483:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829098.582489:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829098.582508:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829098/real 1787829098]  req@ffff8cd0c3da9680 x1874671626905216/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829123 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787829098.582520:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626905600/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829098.582526:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626905216/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829123 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829098.582548:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829098.582552:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829098.582556:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626905216:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829098.582577:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626905344:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829098.582595:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829098.582603:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829098/real 1787829098]  req@ffff8cd0c3da83c0 x1874671626905344/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829123 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829098.582611:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626905344/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829123 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829098.582617:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829098.582619:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829098.582628:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626905344:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829098.582632:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626905472:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829098.582648:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829098.582655:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829098/real 1787829098]  req@ffff8cd0c3da8b40 x1874671626905472/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829123 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829098.582662:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8b40 x1874671626905472/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829123 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829098.582667:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829098.582669:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829098.582671:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626905472:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829098.582674:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626905600:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829098.582684:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829098.582690:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829098/real 1787829098]  req@ffff8cd0c3da8f00 x1874671626905600/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829123 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829098.582697:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626905600/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829123 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829098.582702:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829098.582703:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829098.582705:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626905600:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829098.774146:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829099.158294:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829099.798151:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829100.182204:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829100.822131:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829101.206226:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829101.846123:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829102.230210:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829102.870157:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829103.254246:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829103.702186:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829103.702212:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829103.702217:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829103.702228:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2989
00000100:00080000:2.0:1787829103.702240:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829103.702246:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829103.702296:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626905728/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829103.702325:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829103.702329:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829103.702331:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829103.702336:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2989
00000100:00080000:2.0:1787829103.702340:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829103.702359:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787829103.702363:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626905728:192.168.203.145@tcp:250:kworker.0
00000008:00100000:2.0:1787829103.702369:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829103.702382:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626905856/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829103.702391:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829103.702394:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829103.702396:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829103.702398:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2989
00000100:00080000:2.0:1787829103.702410:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829103.702413:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829103.702416:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787829103.702422:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787829103.702427:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626905984/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829103.702435:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829103.702437:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829103.702439:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829103.702442:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2989
00000100:00080000:2.0:1787829103.702444:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829103.702446:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829103.702450:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829103.702459:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829103/real 1787829103]  req@ffff8cd0c3daa940 x1874671626905728/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829128 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787829103.702460:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626906112/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829103.702502:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626905728/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829128 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829103.702510:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829103.702513:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829103.702516:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626905728:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829103.702533:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626905856:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829103.702548:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829103.702556:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829103/real 1787829103]  req@ffff8cd0c3da8780 x1874671626905856/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829128 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829103.702564:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626905856/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829128 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829103.702570:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829103.702572:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829103.702574:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626905856:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829103.702586:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626905984:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829103.702601:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829103.702608:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829103/real 1787829103]  req@ffff8cd0c3da9a40 x1874671626905984/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829128 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829103.702615:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626905984/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829128 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829103.702620:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829103.702622:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829103.702624:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626905984:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829103.702628:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626906112:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829103.702638:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829103.702644:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829103/real 1787829103]  req@ffff8cd0c3dab0c0 x1874671626906112/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829128 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829103.702651:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626906112/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829128 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829103.702655:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829103.702657:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829103.702659:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626906112:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829103.894121:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829104.278208:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829104.918162:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829105.302274:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829105.942182:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829106.326229:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829106.966208:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829107.350249:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829107.990183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829108.374202:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829108.822196:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829108.822203:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829108.822207:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829108.822215:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2994
00000100:00080000:1.0:1787829108.822225:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829108.822230:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829108.822266:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626906240/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829108.822293:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829108.822314:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829108.822316:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829108.822322:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2994
00000100:00080000:1.0:1787829108.822327:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829108.822332:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829108.822342:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829108.822345:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626906240:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829108.822359:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626906368/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829108.822370:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829108.822373:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829108.822374:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829108.822377:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2994
00000100:00080000:1.0:1787829108.822381:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829108.822383:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829108.822387:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829108.822396:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626906496/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829108.822402:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829108.822405:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829108.822406:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829108.822409:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2994
00000100:00080000:1.0:1787829108.822411:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1787829108.822417:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787829108.822418:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829108.822427:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829108.822436:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626906624/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829108.822480:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829108/real 1787829108]  req@ffff8cd0d48f4b40 x1874671626906240/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829133 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829108.822500:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626906240/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829133 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829108.822511:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829108.822515:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829108.822519:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626906240:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829108.822543:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626906368:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829108.822568:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829108.822581:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829108/real 1787829108]  req@ffff8cd0d48f6580 x1874671626906368/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829133 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829108.822594:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626906368/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829133 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829108.822603:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829108.822606:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829108.822610:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626906368:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829108.822617:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626906496:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829108.822651:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829108.822664:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829108/real 1787829108]  req@ffff8cd0d48f7480 x1874671626906496/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829133 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829108.822678:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626906496/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829133 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829108.822686:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829108.822690:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829108.822693:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626906496:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829108.822700:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626906624:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829108.822722:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829108.822734:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829108/real 1787829108]  req@ffff8cd0d48f4000 x1874671626906624/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829133 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829108.822747:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626906624/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829133 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829108.822755:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829108.822759:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829108.822763:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626906624:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829109.014145:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829109.398199:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829110.038233:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829110.422220:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829111.062206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829111.446261:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829112.086135:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829112.470255:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829113.110176:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829113.494266:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829113.942244:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829113.942270:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829113.942274:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829113.942284:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 2999
00000100:00080000:2.0:1787829113.942295:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829113.942303:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829113.942334:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626906752/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829113.942360:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829113.942365:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829113.942367:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829113.942371:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2999
00000100:00080000:2.0:1787829113.942375:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829113.942380:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829113.942389:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829113.942401:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626906880/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829113.942409:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829113.942411:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829113.942413:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829113.942415:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 2999
00000100:00080000:2.0:1787829113.942421:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829113.942425:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829113.942429:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829113.942435:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626907008/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829113.942450:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829113.942455:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829113.942458:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1787829113.942458:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626906752:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787829113.942476:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 2999
00000100:00080000:2.0:1787829113.942479:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829113.942482:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829113.942486:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829113.942499:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626907136/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829113.942592:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626906880:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829113.942630:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626907008:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829113.942648:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626907136:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829114.134187:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829114.518183:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829115.158164:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829115.542301:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829116.182166:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829116.566196:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:1.0:1787829117.014340:0:2346:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:1.0:1787829117.014378:0:2346:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1787829117.014391:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787829117.014436:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787829117.014446:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787829117.014454:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:3.0:1787829117.014493:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829113/real 1787829117]  req@ffff8cd0c3dabc00 x1874671626907008/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829138 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829117.014520:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626907008/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829138 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829117.014531:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829117.014536:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829117.014540:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626907008:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829117.014548:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829113/real 1787829117]  req@ffff8cd0c3daa580 x1874671626907136/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829138 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829117.014561:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626907136/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829138 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829117.014571:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829117.014588:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829117.014592:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626907136:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829117.014600:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829113/real 1787829117]  req@ffff8cd0c3dab840 x1874671626906752/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829138 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829117.014615:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626906752/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829138 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829117.014624:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829117.014628:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829117.014631:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626906752:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829117.014637:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829113/real 1787829117]  req@ffff8cd0c3da8000 x1874671626906880/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829138 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829117.014648:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626906880/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829138 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829117.014656:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829117.014659:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829117.014662:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626906880:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829117.206167:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829117.590246:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829118.230191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829118.614184:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829119.062249:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829119.062260:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829119.062264:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829119.062274:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3004
00000100:00080000:2.0:1787829119.062285:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829119.062293:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829119.062329:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626907264/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829119.062348:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829119.062351:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829119.062352:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829119.062355:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3004
00000100:00080000:2.0:1787829119.062358:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829119.062361:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787829119.062390:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626907264:192.168.203.145@tcp:250:kworker.0
00000008:00100000:2.0:1787829119.062394:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829119.062412:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626907392/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829119.062424:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829119.062428:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829119.062430:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829119.062434:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3004
00000100:00080000:2.0:1787829119.062446:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829119.062451:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829119.062457:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787829119.062463:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787829119.062479:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626907520/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829119.062490:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829119.062494:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829119.062496:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829119.062501:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3004
00000100:00080000:2.0:1787829119.062505:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829119.062508:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829119.062513:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829119.062530:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daad00 x1874671626907648/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829119.062531:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829119/real 1787829119]  req@ffff8cd0c3da9e00 x1874671626907264/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829144 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829119.062562:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626907264/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829144 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829119.062572:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829119.062577:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829119.062580:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626907264:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829119.062604:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626907392:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829119.062630:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829119.062643:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829119/real 1787829119]  req@ffff8cd0c3da92c0 x1874671626907392/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829144 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829119.062655:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626907392/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829144 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829119.062663:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829119.062665:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829119.062669:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626907392:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829119.062674:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626907520:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829119.062695:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829119.062707:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829119/real 1787829119]  req@ffff8cd0c3daa1c0 x1874671626907520/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829144 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829119.062737:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626907520/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829144 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829119.062758:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829119.062762:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829119.062765:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626907520:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829119.062772:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626907648:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829119.062795:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829119.062807:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829119/real 1787829119]  req@ffff8cd0c3daad00 x1874671626907648/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829144 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829119.062819:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daad00 x1874671626907648/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829144 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829119.062828:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829119.062832:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829119.062835:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626907648:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829119.254160:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829119.638240:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829120.278165:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829120.662219:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829121.302153:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829121.686261:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829122.326171:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829122.710212:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829123.350168:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829123.734220:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829124.182196:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829124.182204:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829124.182207:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829124.182214:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3010
00000100:00080000:1.0:1787829124.182223:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829124.182229:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829124.182259:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626907776/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829124.182279:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829124.182283:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829124.182285:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829124.182289:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3010
00000100:00080000:1.0:1787829124.182292:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829124.182297:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829124.182307:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829124.182335:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626907776:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829124.182338:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626907904/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829124.182353:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829124.182355:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829124.182356:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829124.182359:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3010
00000100:00080000:1.0:1787829124.182363:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829124.182365:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829124.182371:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829124.182381:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626908032/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829124.182388:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829124.182392:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829124.182394:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829124.182398:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3010
00000100:00080000:1.0:1787829124.182404:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829124.182408:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829124.182413:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787829124.182415:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829124.182429:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626908160/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829124.182463:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829124/real 1787829124]  req@ffff8cd0d48f4000 x1874671626907776/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829149 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829124.182490:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626907776/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829149 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829124.182501:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829124.182505:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829124.182509:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626907776:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829124.182531:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626907904:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829124.182556:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829124.182568:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829124/real 1787829124]  req@ffff8cd0d48f7480 x1874671626907904/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829149 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829124.182580:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626907904/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829149 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829124.182588:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829124.182591:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829124.182595:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626907904:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829124.182601:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626908032:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829124.182619:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829124.182630:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829124/real 1787829124]  req@ffff8cd0d48f6580 x1874671626908032/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829149 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829124.182643:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626908032/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829149 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829124.182664:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829124.182669:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829124.182688:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626908032:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829124.182696:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626908160:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829124.182719:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829124.182731:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829124/real 1787829124]  req@ffff8cd0d48f4b40 x1874671626908160/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829149 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829124.182744:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626908160/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829149 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829124.182753:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829124.182757:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829124.182760:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626908160:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829124.374153:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829124.758219:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829125.398184:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829125.782249:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829126.422194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829126.806239:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829127.446214:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829127.830252:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829128.470154:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829128.854165:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787829129.302217:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829129.302228:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787829129.302231:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829129.302240:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3015
00000100:00080000:0.0:1787829129.302249:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829129.302254:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829129.302286:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d2c0 x1874671626908288/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829129.302308:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829129.302311:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829129.302312:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829129.302316:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3015
00000100:00080000:0.0:1787829129.302320:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829129.302325:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829129.302335:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829129.302344:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626908288:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829129.302351:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626908416/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829129.302365:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829129.302370:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787829129.302371:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829129.302381:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3015
00000100:00080000:0.0:1787829129.302386:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829129.302392:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:3.0:1787829129.302399:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:0.0:1787829129.302412:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829129.302424:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626908544/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829129.302432:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829129.302435:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829129.302436:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829129.302439:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3015
00000100:00080000:0.0:1787829129.302442:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1787829129.302446:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829129/real 1787829129]  req@ffff8cd0c8e9d2c0 x1874671626908288/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829154 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829129.302448:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829129.302455:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829129.302467:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626908672/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829129.302505:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d2c0 x1874671626908288/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829154 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829129.302516:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829129.302519:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829129.302522:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626908288:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829129.302541:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626908416:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829129.302560:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829129.302570:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829129/real 1787829129]  req@ffff8cd0c8e9e940 x1874671626908416/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829154 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829129.302579:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626908416/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829154 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829129.302584:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829129.302587:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829129.302589:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626908416:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829129.302593:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626908544:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829129.302605:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829129.302611:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829129/real 1787829129]  req@ffff8cd0c8e9c3c0 x1874671626908544/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829154 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829129.302618:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626908544/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829154 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829129.302655:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829129.302658:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829129.302660:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626908544:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829129.302665:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626908672:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829129.302682:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829129.302690:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829129/real 1787829129]  req@ffff8cd0c8e9d680 x1874671626908672/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829154 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829129.302697:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626908672/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829154 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829129.302703:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829129.302705:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829129.302707:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626908672:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829129.494207:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829129.878207:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829130.518204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829130.902233:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829131.542176:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829131.926236:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829132.566189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829132.950223:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829133.590181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829133.974200:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787829134.422146:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829134.422153:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787829134.422157:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829134.422166:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3020
00000100:00080000:0.0:1787829134.422175:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829134.422180:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829134.422217:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626908800/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829134.422245:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829134.422249:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829134.422266:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829134.422271:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3020
00000100:00080000:0.0:1787829134.422276:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829134.422280:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829134.422289:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829134.422289:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626908800:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829134.422302:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626908928/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829134.422313:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829134.422316:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787829134.422317:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829134.422320:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3020
00000100:00080000:0.0:1787829134.422323:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829134.422327:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829134.422331:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829134.422340:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626909056/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787829134.422347:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787829134.422349:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829134.422352:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829134.422353:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829134.422357:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3020
00000100:00080000:0.0:1787829134.422360:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829134.422363:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829134.422367:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829134.422383:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626909184/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829134.422398:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829134/real 1787829134]  req@ffff8cd0c8e9de00 x1874671626908800/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829159 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829134.422424:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626908800/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829159 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829134.422441:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829134.422444:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829134.422447:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626908800:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829134.422468:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626908928:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829134.422487:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829134.422497:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829134/real 1787829134]  req@ffff8cd0c8e9cf00 x1874671626908928/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829159 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829134.422505:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626908928/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829159 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829134.422510:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829134.422512:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829134.422514:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626908928:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829134.422518:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626909056:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829134.422530:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829134.422536:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829134/real 1787829134]  req@ffff8cd0c8e9fc00 x1874671626909056/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829159 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829134.422544:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626909056/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829159 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829134.422558:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829134.422560:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829134.422562:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626909056:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829134.422566:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626909184:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829134.422579:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829134.422586:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829134/real 1787829134]  req@ffff8cd0c8e9ed00 x1874671626909184/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829159 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829134.422593:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626909184/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829159 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829134.422598:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829134.422599:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829134.422614:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626909184:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829134.614151:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829134.998202:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829135.638166:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829136.022267:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829136.662154:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829137.046207:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829137.686229:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829138.070191:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829138.710138:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829139.094164:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787829139.542195:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829139.542207:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787829139.542211:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829139.542221:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3025
00000100:00080000:0.0:1787829139.542238:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829139.542246:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829139.542284:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626909312/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829139.542318:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829139.542322:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829139.542324:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829139.542329:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3025
00000100:00080000:0.0:1787829139.542332:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829139.542337:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829139.542347:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829139.542363:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626909312:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829139.542364:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626909440/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829139.542383:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829139.542387:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787829139.542403:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829139.542408:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3025
00000100:00080000:0.0:1787829139.542414:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829139.542418:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829139.542425:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787829139.542436:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829139.542442:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626909568/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829139.542455:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829139.542460:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829139.542462:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829139.542467:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3025
00000100:00080000:0.0:1787829139.542472:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00100000:3.0:1787829139.542477:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829139/real 1787829139]  req@ffff8cd0c8e9e580 x1874671626909312/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829164 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829139.542478:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829139.542487:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829139.542510:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626909696/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829139.542511:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626909312/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829164 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829139.542544:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829139.542548:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829139.542552:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626909312:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829139.542589:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626909440:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829139.542610:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829139.542619:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829139/real 1787829139]  req@ffff8cd0c8e9ed00 x1874671626909440/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829164 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829139.542629:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626909440/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829164 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829139.542635:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829139.542637:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829139.542639:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626909440:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829139.542643:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626909568:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829139.542655:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829139.542662:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829139/real 1787829139]  req@ffff8cd0c8e9fc00 x1874671626909568/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829164 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829139.542669:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626909568/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829164 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829139.542683:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829139.542685:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829139.542688:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626909568:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829139.542692:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626909696:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829139.542713:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829139.542720:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829139/real 1787829139]  req@ffff8cd0c8e9cf00 x1874671626909696/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829164 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829139.542728:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626909696/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829164 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829139.542733:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829139.542735:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829139.542737:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626909696:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829139.734188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829140.118228:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829140.758159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829141.142188:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829141.782164:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829142.166219:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829142.806206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829143.190207:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829143.830169:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829144.214215:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787829144.662159:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829144.662167:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787829144.662170:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829144.662179:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3030
00000100:00080000:0.0:1787829144.662189:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829144.662195:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829144.662240:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626909824/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829144.662259:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829144.662262:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829144.662263:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829144.662266:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3030
00000100:00080000:0.0:1787829144.662269:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829144.662272:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829144.662280:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829144.662289:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626909952/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829144.662295:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626909824:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787829144.662300:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829144.662304:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787829144.662305:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829144.662307:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3030
00000100:00080000:0.0:1787829144.662309:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829144.662311:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829144.662315:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829144.662322:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626910080/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829144.662329:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829144.662331:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829144.662332:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829144.662340:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3030
00000100:00080000:0.0:1787829144.662344:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829144.662347:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829144.662351:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787829144.662355:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829144.662364:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626910208/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829144.662400:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829144/real 1787829144]  req@ffff8cd0c8e9de00 x1874671626909824/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829169 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829144.662422:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626909824/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829169 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829144.662431:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829144.662434:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829144.662438:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626909824:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829144.662457:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626909952:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829144.662475:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829144.662484:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829144/real 1787829144]  req@ffff8cd0c8e9d680 x1874671626909952/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829169 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829144.662492:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626909952/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829169 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829144.662498:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829144.662500:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829144.662502:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626909952:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829144.662506:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626910080:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829144.662517:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829144.662523:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829144/real 1787829144]  req@ffff8cd0c8e9c3c0 x1874671626910080/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829169 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829144.662542:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626910080/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829169 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829144.662559:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829144.662578:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829144.662581:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626910080:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829144.662585:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626910208:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829144.662602:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829144.662610:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829144/real 1787829144]  req@ffff8cd0c8e9e940 x1874671626910208/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829169 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829144.662618:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626910208/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829169 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829144.662623:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829144.662626:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829144.662628:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626910208:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829144.854194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829145.238215:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829145.878183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829146.262211:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829146.902180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829147.286201:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829147.926152:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829148.310234:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829148.950179:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829149.334197:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829149.782199:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829149.782207:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829149.782233:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829149.782243:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3035
00000100:00080000:2.0:1787829149.782252:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829149.782259:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829149.782292:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daad00 x1874671626910336/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829149.782314:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829149.782317:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829149.782318:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829149.782321:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3035
00000100:00080000:2.0:1787829149.782323:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829149.782327:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829149.782336:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829149.782345:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626910336:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787829149.782348:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626910464/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829149.782359:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829149.782362:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829149.782363:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829149.782366:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3035
00000100:00080000:2.0:1787829149.782369:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829149.782371:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829149.782375:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829149.782383:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626910592/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829149.782400:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829149.782404:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000800:00000100:3.0:1787829149.782405:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787829149.782407:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829149.782413:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3035
00000100:00080000:2.0:1787829149.782418:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829149.782422:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829149.782427:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829149.782437:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626910720/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829149.782445:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829149/real 1787829149]  req@ffff8cd0c3daad00 x1874671626910336/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829174 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829149.782474:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daad00 x1874671626910336/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829174 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829149.782482:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829149.782485:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829149.782488:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626910336:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829149.782524:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626910464:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829149.782552:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829149.782563:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829149/real 1787829149]  req@ffff8cd0c3daa1c0 x1874671626910464/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829174 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829149.782572:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626910464/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829174 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829149.782579:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829149.782581:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829149.782583:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626910464:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829149.782588:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626910592:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829149.782601:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829149.782608:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829149/real 1787829149]  req@ffff8cd0c3da92c0 x1874671626910592/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829174 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829149.782616:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626910592/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829174 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829149.782631:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829149.782634:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829149.782636:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626910592:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829149.782640:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626910720:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829149.782655:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829149.782662:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829149/real 1787829149]  req@ffff8cd0c3da9e00 x1874671626910720/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829174 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829149.782670:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626910720/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829174 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829149.782676:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829149.782677:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829149.782680:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626910720:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829149.974167:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829150.358211:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829150.998205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829151.382244:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829152.022161:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829152.406295:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829153.046215:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829153.430219:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829154.070197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829154.454245:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829154.902178:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829154.902186:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829154.902190:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829154.902210:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3040
00000100:00080000:1.0:1787829154.902219:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829154.902240:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829154.902269:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626910848/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829154.902292:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829154.902294:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829154.902296:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829154.902299:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3040
00000100:00080000:1.0:1787829154.902301:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829154.902304:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829154.902312:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829154.902322:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626910976/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829154.902332:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787829154.902333:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626910848:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787829154.902339:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829154.902342:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829154.902346:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3040
00000100:00080000:1.0:1787829154.902348:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829154.902351:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829154.902354:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829154.902363:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626911104/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829154.902381:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829154.902384:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829154.902386:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829154.902390:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3040
00000100:00080000:1.0:1787829154.902394:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1787829154.902397:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787829154.902402:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829154.902407:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829154.902419:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626911232/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829154.902442:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829154/real 1787829154]  req@ffff8cd0d48f7840 x1874671626910848/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829179 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829154.902501:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626910848/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829179 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829154.902510:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829154.902513:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829154.902516:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626910848:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829154.902534:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626910976:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829154.902553:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829154.902563:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829154/real 1787829154]  req@ffff8cd0d48f5a40 x1874671626910976/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829179 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829154.902571:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626910976/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829179 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829154.902577:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829154.902579:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829154.902581:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626910976:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829154.902585:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626911104:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829154.902595:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829154.902601:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829154/real 1787829154]  req@ffff8cd0d48f43c0 x1874671626911104/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829179 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829154.902609:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626911104/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829179 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829154.902623:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829154.902625:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829154.902628:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626911104:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829154.902632:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626911232:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829154.902645:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829154.902652:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829154/real 1787829154]  req@ffff8cd0d48f70c0 x1874671626911232/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829179 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829154.902659:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626911232/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829179 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829154.902664:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829154.902666:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829154.902668:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626911232:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787829155.094166:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829155.478178:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787829156.118191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829156.502269:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787829157.142196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829157.526277:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787829158.166183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829158.550281:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787829159.190185:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829159.574239:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829160.022261:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829160.022273:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829160.022278:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829160.022288:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3045
00000100:00080000:2.0:1787829160.022301:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829160.022308:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829160.022364:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626911360/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829160.022401:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829160.022404:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829160.022440:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829160.022447:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3045
00000100:00080000:2.0:1787829160.022451:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829160.022455:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829160.022476:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829160.022491:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626911488/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829160.022498:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626911360:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787829160.022506:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829160.022509:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829160.022512:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829160.022515:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3045
00000100:00080000:2.0:1787829160.022518:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829160.022521:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829160.022527:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829160.022536:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626911616/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829160.022543:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829160.022546:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829160.022548:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829160.022552:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3045
00000100:00080000:2.0:1787829160.022562:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829160.022566:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829160.022570:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787829160.022572:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787829160.022587:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626911744/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829160.022642:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829160/real 1787829160]  req@ffff8cd0c3da8000 x1874671626911360/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829185 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829160.022657:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626911360/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829185 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829160.022665:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829160.022680:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829160.022683:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626911360:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829160.022704:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626911488:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829160.022723:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829160.022732:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829160/real 1787829160]  req@ffff8cd0c3dab840 x1874671626911488/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829185 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829160.022759:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626911488/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829185 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829160.022765:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829160.022768:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829160.022770:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626911488:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829160.022775:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626911616:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829160.022789:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829160.022795:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829160/real 1787829160]  req@ffff8cd0c3daa580 x1874671626911616/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829185 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829160.022804:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626911616/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829185 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829160.022809:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829160.022811:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829160.022813:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626911616:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829160.022825:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626911744:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829160.022847:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829160.022855:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829160/real 1787829160]  req@ffff8cd0c3dabc00 x1874671626911744/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829185 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829160.022869:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626911744/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829185 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829160.022885:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829160.022890:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829160.022894:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626911744:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787829160.214215:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829160.598290:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787829161.238171:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829161.622195:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787829162.262245:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829162.646276:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787829163.286188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829163.670193:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787829164.310207:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829164.694278:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829165.142229:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829165.142241:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829165.142245:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829165.142256:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3050
00000100:00080000:1.0:1787829165.142267:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829165.142275:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829165.142312:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626911872/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829165.142348:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829165.142352:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829165.142354:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829165.142358:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3050
00000100:00080000:1.0:1787829165.142362:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829165.142367:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787829165.142404:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626911872:192.168.203.145@tcp:250:kworker.0
00000008:00100000:1.0:1787829165.142406:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829165.142428:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626912000/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829165.142443:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829165.142447:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829165.142449:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829165.142453:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3050
00000100:00080000:1.0:1787829165.142459:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829165.142464:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:3.0:1787829165.142469:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:1.0:1787829165.142474:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829165.142490:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626912128/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829165.142501:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829165.142505:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:3.0:1787829165.142507:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829165/real 1787829165]  req@ffff8cd0d48f61c0 x1874671626911872/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829190 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829165.142508:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829165.142523:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3050
00000100:00100000:3.0:1787829165.142527:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626911872/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829190 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829165.142530:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829165.142540:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00080000:3.0:1787829165.142541:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829165.142549:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000008:00100000:1.0:1787829165.142552:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829165.142554:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626911872:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829165.142570:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626912256/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829165.142581:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626912000:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829165.142611:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829165.142634:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829165/real 1787829165]  req@ffff8cd0d48f6d00 x1874671626912000/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829190 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829165.142644:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626912000/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829190 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829165.142650:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829165.142652:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829165.142655:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626912000:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829165.142659:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626912128:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829165.142675:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829165.142683:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829165/real 1787829165]  req@ffff8cd0d48f52c0 x1874671626912128/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829190 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829165.142695:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626912128/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829190 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829165.142703:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829165.142707:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829165.142711:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626912128:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829165.142728:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626912256:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829165.142748:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829165.142760:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829165/real 1787829165]  req@ffff8cd0d48f7c00 x1874671626912256/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829190 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829165.142773:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626912256/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829190 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829165.142788:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829165.142792:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829165.142796:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626912256:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787829165.334176:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829165.718169:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787829166.358179:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829166.742461:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787829167.382148:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829167.766189:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:2.0:1787829168.406141:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829168.790229:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829169.430170:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829169.814243:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829170.262328:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829170.262371:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829170.262426:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829170.262467:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3056
00000100:00080000:1.0:1787829170.262489:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829170.262512:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829170.262606:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626912384/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829170.262644:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829170.262648:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829170.262650:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829170.262659:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3056
00000100:00080000:1.0:1787829170.262664:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829170.262669:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829170.262683:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829170.262698:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626912512/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829170.262713:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787829170.262713:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626912384:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787829170.262724:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829170.262728:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829170.262732:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3056
00000100:00080000:1.0:1787829170.262736:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829170.262739:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829170.262744:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829170.262757:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626912640/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829170.262789:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829170.262810:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829170.262812:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829170.262816:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3056
00000100:00080000:1.0:1787829170.262820:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829170.262823:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829170.262828:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829170.262839:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626912768/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787829170.262852:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829170.262935:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829170/real 1787829170]  req@ffff8cd0d48f7c00 x1874671626912384/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829195 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829170.262955:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626912384/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829195 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829170.262967:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829170.262971:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829170.262976:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626912384:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829170.263004:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626912512:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829170.263027:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829170.263039:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829170/real 1787829170]  req@ffff8cd0d48f52c0 x1874671626912512/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829195 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829170.263050:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626912512/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829195 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829170.263073:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829170.263077:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829170.263080:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626912512:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829170.263086:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626912640:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829170.263180:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829170.263193:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829170/real 1787829170]  req@ffff8cd0d48f6d00 x1874671626912640/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829195 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829170.263204:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626912640/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829195 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829170.263212:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829170.263215:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829170.263217:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626912640:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829170.263223:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626912768:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829170.263278:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829170.263289:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829170/real 1787829170]  req@ffff8cd0d48f61c0 x1874671626912768/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829195 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829170.263299:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626912768/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829195 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829170.263306:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829170.263309:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829170.263317:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626912768:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829170.454171:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829170.838415:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829171.478175:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829171.862209:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829172.502193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829172.886296:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829173.526185:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829173.910260:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829174.550170:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829174.934243:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787829175.382213:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829175.382222:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787829175.382237:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829175.382246:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3061
00000100:00080000:0.0:1787829175.382255:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829175.382261:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829175.382293:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d2c0 x1874671626912896/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829175.382355:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829175.382358:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829175.382360:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829175.382374:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3061
00000100:00080000:0.0:1787829175.382377:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829175.382380:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829175.382388:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829175.382398:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626913024/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829175.382407:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829175.382409:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787829175.382411:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829175.382413:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3061
00000100:00080000:0.0:1787829175.382415:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829175.382418:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829175.382422:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829175.382430:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626913152/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829175.382440:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829175.382446:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829175.382448:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:3.0:1787829175.382448:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626912896:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787829175.382458:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3061
00000100:00080000:0.0:1787829175.382467:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829175.382471:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829175.382477:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829175.382495:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626913280/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787829175.382537:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829175.382585:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829175/real 1787829175]  req@ffff8cd0c8e9d2c0 x1874671626912896/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829200 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829175.382600:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d2c0 x1874671626912896/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829200 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829175.382610:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829175.382613:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829175.382616:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626912896:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829175.382622:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626913024:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829175.382637:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829175.382646:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829175/real 1787829175]  req@ffff8cd0c8e9f480 x1874671626913024/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829200 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829175.382653:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626913024/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829200 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829175.382676:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829175.382680:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829175.382682:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626913024:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829175.382709:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626913152:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829175.382726:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829175.382733:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829175/real 1787829175]  req@ffff8cd0c8e9f840 x1874671626913152/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829200 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829175.382741:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626913152/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829200 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829175.382746:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829175.382748:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829175.382750:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626913152:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829175.382755:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626913280:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829175.382775:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829175.382787:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829175/real 1787829175]  req@ffff8cd0c8e9c000 x1874671626913280/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829200 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829175.382800:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626913280/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829200 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829175.382809:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829175.382813:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829175.382824:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626913280:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829175.574234:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829175.958209:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829176.598168:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829176.982222:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829177.622148:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829178.006191:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829178.646162:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829179.030305:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829179.670153:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829180.054200:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787829180.502183:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829180.502202:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787829180.502205:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829180.502213:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3066
00000100:00080000:0.0:1787829180.502223:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829180.502228:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829180.502260:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626913408/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829180.502298:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829180.502301:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829180.502302:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829180.502305:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3066
00000100:00080000:0.0:1787829180.502308:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829180.502312:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829180.502320:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829180.502342:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626913536/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829180.502356:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626913408:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787829180.502374:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829180.502381:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787829180.502383:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829180.502387:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3066
00000100:00080000:0.0:1787829180.502390:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829180.502393:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829180.502397:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829180.502407:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626913664/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829180.502413:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829180.502415:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829180.502416:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829180.502419:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3066
00000100:00080000:0.0:1787829180.502428:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829180.502431:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829180.502434:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829180.502441:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626913792/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829180.502492:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626913536:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829180.502519:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626913664:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829180.502548:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626913792:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829180.694191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829181.078244:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829181.718214:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829182.102288:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829182.742191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829183.126223:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:1.0:1787829183.574379:0:2343:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:1.0:1787829183.574422:0:2343:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1787829183.574439:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787829183.574475:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787829183.574485:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787829183.574493:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:3.0:1787829183.574675:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829180/real 1787829183]  req@ffff8cd0c8e9c000 x1874671626913536/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829205 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829183.574702:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626913536/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829205 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829183.574716:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829183.574721:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829183.574725:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626913536:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829183.574749:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829180/real 1787829183]  req@ffff8cd0c8e9f840 x1874671626913664/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829205 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829183.574764:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626913664/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829205 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829183.574773:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829183.574776:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829183.574779:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626913664:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829183.574786:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829180/real 1787829183]  req@ffff8cd0c8e9f480 x1874671626913792/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829205 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829183.574809:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626913792/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829205 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829183.574816:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829183.574820:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829183.574823:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626913792:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829183.574829:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829180/real 1787829183]  req@ffff8cd0c8e9cb40 x1874671626913408/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829205 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829183.574841:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626913408/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829205 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829183.574848:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829183.574850:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829183.574853:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626913408:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829183.766178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829184.150213:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829184.790193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829185.174309:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787829185.622168:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829185.622179:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787829185.622203:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829185.622214:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3071
00000100:00080000:0.0:1787829185.622227:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829185.622234:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829185.622293:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d2c0 x1874671626913920/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829185.622329:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829185.622333:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829185.622336:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829185.622342:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3071
00000100:00080000:0.0:1787829185.622348:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829185.622354:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829185.622367:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829185.622383:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626914048/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829185.622400:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626913920:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787829185.622401:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829185.622410:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787829185.622412:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829185.622416:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3071
00000100:00080000:0.0:1787829185.622421:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829185.622425:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829185.622432:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829185.622447:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626914176/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829185.622473:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829185.622478:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829185.622481:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829185.622486:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3071
00000100:00080000:0.0:1787829185.622497:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:3.0:1787829185.622501:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787829185.622506:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829185.622514:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829185.622529:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626914304/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829185.622566:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829185/real 1787829185]  req@ffff8cd0c8e9d2c0 x1874671626913920/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829210 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829185.622589:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d2c0 x1874671626913920/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829210 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829185.622603:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829185.622608:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829185.622613:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626913920:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829185.622646:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626914048:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829185.622675:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829185.622690:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829185/real 1787829185]  req@ffff8cd0c8e9e940 x1874671626914048/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829210 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829185.622703:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626914048/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829210 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829185.622711:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829185.622714:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829185.622718:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626914048:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829185.622725:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626914176:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829185.622761:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829185.622775:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829185/real 1787829185]  req@ffff8cd0c8e9c3c0 x1874671626914176/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829210 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829185.622788:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626914176/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829210 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829185.622796:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829185.622799:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829185.622802:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626914176:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829185.622809:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626914304:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829185.622837:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829185.622849:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829185/real 1787829185]  req@ffff8cd0c8e9d680 x1874671626914304/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829210 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829185.622860:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626914304/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829210 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829185.622868:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829185.622872:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829185.622875:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626914304:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829185.814288:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829186.198198:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829186.838159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829187.222277:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829187.862183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829188.246192:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829188.886147:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829189.270258:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829189.910278:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829190.294304:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829190.742352:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829190.742371:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829190.742379:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829190.742398:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3076
00000100:00080000:1.0:1787829190.742419:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829190.742433:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829190.742516:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626914432/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829190.742590:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829190.742593:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829190.742595:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829190.742599:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3076
00000100:00080000:1.0:1787829190.742610:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829190.742615:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829190.742627:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829190.742637:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626914432:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829190.742643:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626914560/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829190.742653:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829190.742656:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829190.742658:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829190.742661:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3076
00000100:00080000:1.0:1787829190.742664:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829190.742666:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829190.742671:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829190.742679:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626914688/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829190.742703:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829190.742706:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829190.742707:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829190.742709:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3076
00000100:00080000:1.0:1787829190.742718:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829190.742722:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829190.742726:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829190.742734:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626914816/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787829190.742771:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829190.742829:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829190/real 1787829190]  req@ffff8cd0d48f70c0 x1874671626914432/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829215 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829190.742848:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626914432/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829215 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829190.742860:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829190.742864:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829190.742867:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626914432:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829190.742908:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626914560:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829190.742936:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829190.742986:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829190/real 1787829190]  req@ffff8cd0d48f43c0 x1874671626914560/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829215 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829190.742996:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626914560/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829215 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829190.743003:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829190.743005:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829190.743008:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626914560:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829190.743012:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626914688:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829190.743030:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829190.743037:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829190/real 1787829190]  req@ffff8cd0d48f5a40 x1874671626914688/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829215 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829190.743072:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626914688/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829215 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829190.743078:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829190.743080:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829190.743083:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626914688:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829190.743087:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626914816:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829190.743102:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829190.743109:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829190/real 1787829190]  req@ffff8cd0d48f7840 x1874671626914816/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829215 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829190.743140:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626914816/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829215 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829190.743147:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829190.743149:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829190.743152:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626914816:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829190.934252:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829191.318286:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829191.958232:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829192.342197:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829192.982261:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829193.366271:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829194.006242:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829194.390164:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829195.030158:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829195.414425:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787829195.862317:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829195.862331:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787829195.862337:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829195.862352:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3081
00000100:00080000:0.0:1787829195.862384:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829195.862393:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829195.862443:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626914944/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829195.862497:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829195.862501:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829195.862516:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829195.862519:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3081
00000100:00080000:0.0:1787829195.862523:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829195.862527:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829195.862539:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829195.862553:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626915072/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829195.862554:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626914944:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787829195.862563:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829195.862570:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787829195.862572:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829195.862575:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3081
00000100:00080000:0.0:1787829195.862577:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829195.862580:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829195.862583:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829195.862592:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626915200/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829195.862600:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829195.862603:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829195.862604:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829195.862607:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3081
00000100:00080000:0.0:1787829195.862609:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829195.862611:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829195.862615:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829195.862632:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626915328/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787829195.862643:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829195.862688:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829195/real 1787829195]  req@ffff8cd0c8e9de00 x1874671626914944/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829220 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829195.862703:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626914944/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829220 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829195.862713:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829195.862716:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829195.862719:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626914944:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829195.862746:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626915072:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829195.862761:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829195.862770:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829195/real 1787829195]  req@ffff8cd0c8e9cf00 x1874671626915072/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829220 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829195.862778:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626915072/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829220 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829195.862784:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829195.862786:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829195.862789:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626915072:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829195.862792:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626915200:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829195.862803:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829195.862809:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829195/real 1787829195]  req@ffff8cd0c8e9fc00 x1874671626915200/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829220 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829195.862828:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626915200/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829220 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829195.862834:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829195.862836:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829195.862838:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626915200:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829195.862842:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626915328:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829195.862855:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829195.862861:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829195/real 1787829195]  req@ffff8cd0c8e9ed00 x1874671626915328/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829220 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829195.862868:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626915328/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829220 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829195.862874:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829195.862875:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829195.862877:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626915328:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829196.054148:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829196.438253:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829197.078196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829197.462336:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829198.102247:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829198.486253:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829199.126164:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829199.510279:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829200.150187:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829200.534220:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829200.982222:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829200.982230:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829200.982234:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829200.982242:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3086
00000100:00080000:1.0:1787829200.982252:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829200.982257:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829200.982291:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626915456/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829200.982330:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829200.982333:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829200.982335:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829200.982338:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3086
00000100:00080000:1.0:1787829200.982341:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829200.982344:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829200.982353:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829200.982363:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626915584/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829200.982388:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787829200.982404:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626915456:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787829200.982405:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829200.982408:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829200.982413:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3086
00000100:00080000:1.0:1787829200.982416:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829200.982419:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829200.982423:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829200.982432:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626915712/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829200.982440:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829200.982442:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829200.982444:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829200.982447:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3086
00000100:00080000:1.0:1787829200.982449:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829200.982451:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829200.982454:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829200.982464:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626915840/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787829200.982490:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829200.982528:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829200/real 1787829200]  req@ffff8cd0d48f4b40 x1874671626915456/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829225 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829200.982543:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626915456/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829225 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829200.982551:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829200.982554:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829200.982557:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626915456:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829200.982563:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626915584:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829200.982579:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829200.982587:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829200/real 1787829200]  req@ffff8cd0d48f6580 x1874671626915584/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829225 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829200.982595:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626915584/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829225 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829200.982601:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829200.982603:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829200.982605:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626915584:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829200.982624:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626915712:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829200.982636:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829200.982642:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829200/real 1787829200]  req@ffff8cd0d48f7480 x1874671626915712/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829225 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829200.982660:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626915712/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829225 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829200.982666:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829200.982668:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829200.982670:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626915712:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829200.982674:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626915840:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829200.982687:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829200.982693:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829200/real 1787829200]  req@ffff8cd0d48f4000 x1874671626915840/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829225 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829200.982701:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626915840/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829225 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829200.982706:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829200.982708:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829200.982716:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626915840:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829201.174161:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829201.558261:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829202.198190:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829202.582222:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829203.222213:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829203.606218:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829204.246164:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829204.630180:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829205.270178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829205.654246:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829206.102310:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829206.102322:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829206.102346:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829206.102359:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3091
00000100:00080000:2.0:1787829206.102372:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829206.102382:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829206.102429:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626915968/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829206.102476:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829206.102482:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829206.102485:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829206.102491:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3091
00000100:00080000:2.0:1787829206.102497:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829206.102504:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829206.102543:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829206.102557:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626915968:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787829206.102584:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626916096/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829206.102603:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829206.102608:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829206.102611:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829206.102616:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3091
00000100:00080000:2.0:1787829206.102624:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829206.102630:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829206.102638:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787829206.102639:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787829206.102675:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626916224/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829206.102690:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:3.0:1787829206.102696:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829206/real 1787829206]  req@ffff8cd0c3dab0c0 x1874671626915968/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829231 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829206.102714:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829206.102719:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829206.102729:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3091
00000100:00100000:3.0:1787829206.102736:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626915968/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829231 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829206.102743:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829206.102746:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829206.102752:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00080000:2.0:1787829206.102753:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787829206.102757:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626915968:192.168.203.145@tcp:250:kworker.0
00000008:00100000:2.0:1787829206.102764:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829206.102780:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626916352/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829206.102788:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626916096:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829206.102816:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829206.102834:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829206/real 1787829206]  req@ffff8cd0c3da9a40 x1874671626916096/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829231 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829206.102850:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626916096/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829231 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829206.102868:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829206.102872:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829206.102876:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626916096:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829206.102882:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626916224:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829206.102905:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829206.102917:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829206/real 1787829206]  req@ffff8cd0c3da8780 x1874671626916224/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829231 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829206.102946:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626916224/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829231 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829206.102952:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829206.102954:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829206.102957:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626916224:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829206.102969:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626916352:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829206.102993:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829206.103001:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829206/real 1787829206]  req@ffff8cd0c3daa940 x1874671626916352/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829231 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829206.103025:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626916352/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829231 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829206.103033:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829206.103036:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829206.103039:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626916352:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829206.294173:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829206.678223:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829207.318287:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829207.702275:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829208.342163:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829208.726177:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829209.366202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829209.750225:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829210.390210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829210.774340:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829211.222200:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829211.222209:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829211.222213:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829211.222227:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3097
00000100:00080000:1.0:1787829211.222239:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829211.222245:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829211.222277:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626916480/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829211.222310:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829211.222312:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829211.222313:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829211.222316:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3097
00000100:00080000:1.0:1787829211.222319:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829211.222322:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829211.222329:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829211.222339:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626916608/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829211.222348:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829211.222351:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829211.222353:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829211.222355:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3097
00000100:00080000:1.0:1787829211.222357:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829211.222359:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829211.222362:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829211.222369:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626916736/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829211.222377:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829211.222380:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829211.222381:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829211.222388:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3097
00000100:00100000:3.0:1787829211.222390:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626916480:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787829211.222400:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829211.222405:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829211.222409:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829211.222420:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626916864/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:3.0:1787829211.222495:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829211.222566:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829211/real 1787829211]  req@ffff8cd0d48f4000 x1874671626916480/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829236 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829211.222592:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626916480/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829236 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829211.222606:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829211.222611:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829211.222617:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626916480:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829211.222627:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626916608:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829211.222650:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829211.222660:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829211/real 1787829211]  req@ffff8cd0d48f7480 x1874671626916608/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829236 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829211.222669:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626916608/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829236 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829211.222676:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829211.222678:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829211.222681:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626916608:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829211.222705:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626916736:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829211.222719:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829211.222725:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829211/real 1787829211]  req@ffff8cd0d48f6580 x1874671626916736/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829236 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829211.222744:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626916736/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829236 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829211.222750:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829211.222753:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829211.222755:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626916736:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829211.222759:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626916864:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829211.222772:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829211.222779:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829211/real 1787829211]  req@ffff8cd0d48f4b40 x1874671626916864/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829236 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829211.222789:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626916864/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829236 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829211.222798:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829211.222801:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829211.222805:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626916864:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829211.414215:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829211.798261:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829212.438186:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829212.822226:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829213.462170:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829213.846202:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829214.486165:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829214.870229:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829215.510215:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829215.894222:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829216.342186:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829216.342214:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829216.342219:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829216.342230:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3102
00000100:00080000:1.0:1787829216.342241:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829216.342249:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829216.342315:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626916992/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829216.342365:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829216.342370:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829216.342372:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829216.342376:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3102
00000100:00080000:1.0:1787829216.342380:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829216.342391:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829216.342402:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829216.342410:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626916992:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829216.342416:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626917120/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829216.342426:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829216.342437:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829216.342438:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829216.342442:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3102
00000100:00080000:1.0:1787829216.342445:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829216.342448:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829216.342453:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829216.342465:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626917248/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829216.342476:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829216.342480:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829216.342482:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829216.342486:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3102
00000100:00080000:1.0:1787829216.342500:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829216.342506:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:3.0:1787829216.342513:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:1.0:1787829216.342515:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829216.342528:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626917376/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829216.342567:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829216/real 1787829216]  req@ffff8cd0d48f7840 x1874671626916992/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829241 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829216.342584:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626916992/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829241 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829216.342594:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829216.342603:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829216.342607:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626916992:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829216.342630:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626917120:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829216.342649:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829216.342659:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829216/real 1787829216]  req@ffff8cd0d48f5a40 x1874671626917120/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829241 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829216.342674:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626917120/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829241 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829216.342680:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829216.342683:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829216.342685:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626917120:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829216.342689:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626917248:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829216.342702:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829216.342708:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829216/real 1787829216]  req@ffff8cd0d48f43c0 x1874671626917248/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829241 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829216.342727:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626917248/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829241 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829216.342733:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829216.342735:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829216.342737:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626917248:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829216.342741:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626917376:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829216.342754:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829216.342761:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829216/real 1787829216]  req@ffff8cd0d48f70c0 x1874671626917376/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829241 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829216.342768:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626917376/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829241 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829216.342773:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829216.342775:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829216.342777:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626917376:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829216.534197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829216.918204:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829217.558168:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829217.942225:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829218.582169:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829218.966184:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829219.606165:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829219.990198:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829220.630144:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829221.014257:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829221.462156:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829221.462164:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829221.462168:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829221.462176:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3107
00000100:00080000:1.0:1787829221.462186:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829221.462192:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829221.462227:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626917504/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829221.462270:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829221.462273:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829221.462275:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829221.462278:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3107
00000100:00080000:1.0:1787829221.462281:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829221.462285:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829221.462295:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829221.462311:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626917632/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829221.462323:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626917504:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787829221.462343:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829221.462349:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829221.462351:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829221.462357:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3107
00000100:00080000:1.0:1787829221.462362:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829221.462367:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829221.462374:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:3.0:1787829221.462394:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829221.462397:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626917760/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829221.462407:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829221.462410:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829221.462412:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829221.462416:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3107
00000100:00080000:1.0:1787829221.462426:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829221.462429:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829221.462434:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829221.462449:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829221/real 1787829221]  req@ffff8cd0d48f61c0 x1874671626917504/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829246 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829221.462454:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626917888/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829221.462496:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626917504/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829246 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829221.462506:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829221.462509:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829221.462513:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626917504:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829221.462533:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626917632:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829221.462560:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829221.462569:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829221/real 1787829221]  req@ffff8cd0d48f6d00 x1874671626917632/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829246 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829221.462578:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626917632/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829246 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829221.462583:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829221.462586:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829221.462588:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626917632:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829221.462591:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626917760:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829221.462602:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829221.462608:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829221/real 1787829221]  req@ffff8cd0d48f52c0 x1874671626917760/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829246 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829221.462627:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626917760/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829246 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829221.462632:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829221.462634:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829221.462636:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626917760:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829221.462640:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626917888:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829221.462652:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829221.462658:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829221/real 1787829221]  req@ffff8cd0d48f7c00 x1874671626917888/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829246 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829221.462665:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626917888/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829246 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829221.462671:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829221.462672:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829221.462674:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626917888:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829221.654188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829222.038219:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829222.678230:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829223.062190:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829223.702226:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829224.086263:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829224.726230:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829225.110194:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829225.750170:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829226.134234:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829226.582206:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829226.582214:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829226.582218:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829226.582247:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3112
00000100:00080000:2.0:1787829226.582256:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829226.582262:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829226.582293:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626918016/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829226.582318:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829226.582320:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829226.582322:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829226.582324:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3112
00000100:00080000:2.0:1787829226.582326:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829226.582329:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829226.582337:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829226.582346:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8b40 x1874671626918144/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829226.582353:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829226.582356:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829226.582357:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829226.582359:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3112
00000100:00080000:2.0:1787829226.582360:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829226.582362:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829226.582365:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829226.582371:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626918272/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829226.582387:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829226.582389:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829226.582391:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829226.582394:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3112
00000100:00080000:2.0:1787829226.582397:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829226.582400:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829226.582404:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829226.582410:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626918400/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:3.0:1787829226.582445:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626918016:192.168.203.145@tcp:250:kworker.0
00000800:00000100:3.0:1787829226.582564:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829226.582630:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829226/real 1787829226]  req@ffff8cd0c3da8f00 x1874671626918016/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829251 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829226.582657:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626918016/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829251 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829226.582670:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829226.582676:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829226.582681:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626918016:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829226.582691:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626918144:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829226.582718:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829226.582734:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829226/real 1787829226]  req@ffff8cd0c3da8b40 x1874671626918144/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829251 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829226.582749:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8b40 x1874671626918144/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829251 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829226.582759:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829226.582763:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829226.582767:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626918144:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829226.582775:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626918272:192.168.203.145@tcp:8:kworker.0
00000800:00000100:3.0:1787829226.582799:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829226.582812:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829226/real 1787829226]  req@ffff8cd0c3da83c0 x1874671626918272/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829251 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829226.582842:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626918272/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829251 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829226.582851:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829226.582855:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829226.582859:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626918272:192.168.203.145@tcp:8:kworker.0
00000100:00100000:3.0:1787829226.582866:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626918400:192.168.203.145@tcp:38:kworker.0
00000800:00000100:3.0:1787829226.582892:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829226.582906:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829226/real 1787829226]  req@ffff8cd0c3da9680 x1874671626918400/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829251 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829226.582921:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626918400/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829251 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829226.582930:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829226.582934:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:3.0:1787829226.582937:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626918400:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829226.774250:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829227.158202:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829227.798180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829228.182229:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829228.822181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829229.206211:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829229.846164:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829230.230200:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829230.870195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829231.254206:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787829231.702192:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829231.702218:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787829231.702221:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829231.702229:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3117
00000100:00080000:3.0:1787829231.702238:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829231.702244:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787829231.702281:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626918528/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829231.702304:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829231.702307:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787829231.702308:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829231.702311:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3117
00000100:00080000:3.0:1787829231.702318:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829231.702322:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829231.702333:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829231.702345:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626918528:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829231.702345:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626918656/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829231.702362:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829231.702365:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787829231.702366:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829231.702369:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3117
00000100:00080000:3.0:1787829231.702371:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829231.702373:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829231.702377:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829231.702385:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626918784/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829231.702398:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829231.702400:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787829231.702402:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829231.702406:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3117
00000100:00080000:3.0:1787829231.702409:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829231.702412:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829231.702416:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787829231.702417:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829231.702426:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626918912/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787829231.702462:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829231/real 1787829231]  req@ffff8cd0c8eac3c0 x1874671626918528/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829256 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787829231.702481:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626918528/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829256 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829231.702490:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787829231.702495:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787829231.702499:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626918528:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787829231.702523:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626918656:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787829231.702546:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787829231.702558:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829231/real 1787829231]  req@ffff8cd0c8eaf0c0 x1874671626918656/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829256 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787829231.702579:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626918656/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829256 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829231.702588:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787829231.702601:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787829231.702608:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626918656:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787829231.702615:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626918784:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787829231.702636:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787829231.702646:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829231/real 1787829231]  req@ffff8cd0c8eac780 x1874671626918784/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829256 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787829231.702659:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626918784/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829256 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829231.702667:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787829231.702670:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787829231.702673:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626918784:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787829231.702679:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626918912:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787829231.702696:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787829231.702706:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829231/real 1787829231]  req@ffff8cd0c8eada40 x1874671626918912/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829256 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787829231.702718:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626918912/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829256 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829231.702737:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787829231.702741:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787829231.702744:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626918912:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829231.894158:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829232.278282:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829232.918137:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829233.302182:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829233.942136:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829234.326239:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829234.966147:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829235.350260:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829235.990134:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829236.374250:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829236.822226:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829236.822236:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829236.822241:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829236.822252:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3122
00000100:00080000:1.0:1787829236.822264:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829236.822271:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829236.822311:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626919040/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829236.822341:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829236.822346:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829236.822348:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829236.822351:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3122
00000100:00080000:1.0:1787829236.822355:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829236.822360:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829236.822369:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829236.822384:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626919168/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829236.822397:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829236.822414:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829236.822419:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:2.0:1787829236.822419:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626919040:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787829236.822431:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3122
00000100:00080000:1.0:1787829236.822436:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829236.822441:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829236.822447:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829236.822462:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626919296/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829236.822474:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829236.822478:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829236.822480:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829236.822488:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3122
00000800:00000100:2.0:1787829236.822488:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787829236.822496:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829236.822501:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829236.822507:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829236.822520:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626919424/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787829236.822539:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829236/real 1787829236]  req@ffff8cd0d48f7c00 x1874671626919040/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829261 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787829236.822560:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626919040/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829261 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829236.822571:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787829236.822576:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787829236.822580:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626919040:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787829236.822587:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626919168:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787829236.822609:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787829236.822621:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829236/real 1787829236]  req@ffff8cd0d48f52c0 x1874671626919168/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829261 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787829236.822633:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626919168/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829261 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829236.822641:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787829236.822644:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787829236.822647:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626919168:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787829236.822679:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626919296:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787829236.822701:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787829236.822714:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829236/real 1787829236]  req@ffff8cd0d48f6d00 x1874671626919296/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829261 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787829236.822726:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626919296/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829261 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829236.822734:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787829236.822738:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787829236.822741:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626919296:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787829236.822747:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626919424:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787829236.822765:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787829236.822775:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829236/real 1787829236]  req@ffff8cd0d48f61c0 x1874671626919424/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829261 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787829236.822786:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626919424/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829261 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829236.822794:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787829236.822797:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787829236.822800:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626919424:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829237.014134:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829237.398206:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829238.038136:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829238.422312:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829239.062211:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829239.446249:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829240.086134:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829240.470276:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829241.110188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829241.494212:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829241.942208:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829241.942220:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829241.942224:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829241.942251:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3127
00000100:00080000:1.0:1787829241.942261:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829241.942267:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829241.942320:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626919552/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829241.942356:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829241.942361:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829241.942363:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829241.942370:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3127
00000100:00080000:1.0:1787829241.942375:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829241.942381:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829241.942392:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829241.942395:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626919552:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829241.942413:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626919680/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829241.942427:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829241.942430:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829241.942432:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829241.942436:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3127
00000100:00080000:1.0:1787829241.942440:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829241.942443:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829241.942448:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787829241.942458:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829241.942469:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626919808/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829241.942483:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829241.942486:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829241.942488:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829241.942492:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3127
00000100:00080000:1.0:1787829241.942502:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829241.942506:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829241.942512:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829241.942531:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829241/real 1787829241]  req@ffff8cd0d48f70c0 x1874671626919552/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829266 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829241.942540:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626919936/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787829241.942579:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626919552/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829266 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829241.942588:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787829241.942591:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787829241.942596:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626919552:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787829241.942622:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626919680:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787829241.942650:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787829241.942661:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829241/real 1787829241]  req@ffff8cd0d48f43c0 x1874671626919680/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829266 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787829241.942689:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626919680/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829266 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829241.942696:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787829241.942699:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787829241.942702:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626919680:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787829241.942707:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626919808:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787829241.942724:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787829241.942731:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829241/real 1787829241]  req@ffff8cd0d48f5a40 x1874671626919808/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829266 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787829241.942741:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626919808/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829266 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829241.942758:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787829241.942760:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787829241.942763:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626919808:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787829241.942768:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626919936:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787829241.942785:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787829241.942793:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829241/real 1787829241]  req@ffff8cd0d48f7840 x1874671626919936/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829266 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787829241.942802:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626919936/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829266 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829241.942807:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787829241.942809:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787829241.942811:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626919936:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829242.134160:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829242.518243:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829243.158158:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829243.542213:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829244.182237:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829244.566249:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829245.206245:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829245.590231:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829246.230213:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829246.614305:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829247.062260:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829247.062275:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829247.062292:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829247.062307:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3132
00000100:00080000:1.0:1787829247.062324:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829247.062336:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829247.062425:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626920064/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829247.062501:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829247.062506:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829247.062508:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829247.062513:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3132
00000100:00080000:1.0:1787829247.062519:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829247.062527:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829247.062543:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829247.062575:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626920064:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829247.062589:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626920192/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829247.062608:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829247.062654:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829247.062658:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829247.062663:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3132
00000100:00080000:1.0:1787829247.062669:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829247.062675:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829247.062683:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829247.062702:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626920320/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829247.062714:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829247.062718:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829247.062720:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829247.062725:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3132
00000100:00080000:1.0:1787829247.062730:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829247.062734:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829247.062741:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829247.062769:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626920192:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787829247.062778:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626920448/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787829247.062868:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626920320:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787829247.062904:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626920448:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829247.254231:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829247.638184:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829248.278198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829248.662293:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829249.302220:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829249.686222:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:0.0:1787829250.134324:0:2345:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:0.0:1787829250.134361:0:2345:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:0.0:1787829250.134388:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:0.0:1787829250.134435:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:0.0:1787829250.134445:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:0.0:1787829250.134452:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:2.0:1787829250.134524:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829247/real 1787829250]  req@ffff8cd0d48f4000 x1874671626920448/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829272 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787829250.134550:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626920448/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829272 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829250.134571:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787829250.134577:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787829250.134601:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626920448:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787829250.134611:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829247/real 1787829250]  req@ffff8cd0d48f6580 x1874671626920192/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829272 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787829250.134626:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626920192/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829272 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829250.134636:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787829250.134644:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787829250.134648:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626920192:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787829250.134654:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829247/real 1787829250]  req@ffff8cd0d48f7480 x1874671626920320/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829272 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787829250.134667:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626920320/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829272 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829250.134675:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787829250.134678:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787829250.134682:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626920320:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787829250.134688:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829247/real 1787829250]  req@ffff8cd0d48f4b40 x1874671626920064/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829272 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787829250.134701:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626920064/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829272 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829250.134719:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787829250.134723:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787829250.134727:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626920064:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829250.326197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829250.710304:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829251.350174:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829251.734215:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829252.182219:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829252.182235:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829252.182238:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829252.182248:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3138
00000100:00080000:2.0:1787829252.182257:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829252.182263:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829252.182295:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626920576/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829252.182322:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829252.182325:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829252.182328:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829252.182333:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3138
00000100:00080000:2.0:1787829252.182338:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829252.182343:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829252.182353:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829252.182364:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626920576:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787829252.182371:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626920704/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829252.182386:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829252.182390:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829252.182406:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829252.182412:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3138
00000100:00080000:2.0:1787829252.182417:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1787829252.182422:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787829252.182426:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829252.182435:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829252.182452:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8b40 x1874671626920832/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829252.182464:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1787829252.182467:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829252/real 1787829252]  req@ffff8cd0c3da9680 x1874671626920576/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829277 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829252.182468:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829252.182472:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829252.182481:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3138
00000100:00080000:2.0:1787829252.182488:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829252.182493:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829252.182501:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829252.182505:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626920576/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829277 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787829252.182517:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626920960/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829252.182521:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829252.182531:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829252.182540:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626920576:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829252.182580:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626920704:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829252.182609:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829252.182624:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829252/real 1787829252]  req@ffff8cd0c3da83c0 x1874671626920704/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829277 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829252.182638:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626920704/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829277 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829252.182647:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829252.182651:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829252.182654:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626920704:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829252.182673:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626920832:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829252.182700:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829252.182714:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829252/real 1787829252]  req@ffff8cd0c3da8b40 x1874671626920832/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829277 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829252.182729:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8b40 x1874671626920832/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829277 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829252.182738:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829252.182741:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829252.182744:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626920832:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829252.182750:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626920960:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829252.182770:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829252.182780:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829252/real 1787829252]  req@ffff8cd0c3da8f00 x1874671626920960/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829277 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829252.182807:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626920960/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829277 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829252.182815:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829252.182818:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829252.182821:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626920960:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829252.374201:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829252.758211:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829253.398153:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829253.782239:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829254.422157:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829254.806242:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829255.446144:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829255.830218:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829256.470160:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829256.854215:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829257.302172:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829257.302180:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829257.302183:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829257.302192:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3143
00000100:00080000:1.0:1787829257.302201:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829257.302207:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829257.302240:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626921088/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829257.302273:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829257.302276:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829257.302277:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829257.302280:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3143
00000100:00080000:1.0:1787829257.302283:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829257.302286:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829257.302294:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829257.302303:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626921216/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829257.302310:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829257.302312:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829257.302313:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829257.302315:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3143
00000100:00080000:1.0:1787829257.302317:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829257.302319:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829257.302322:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829257.302328:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626921344/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829257.302335:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829257.302338:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829257.302340:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829257.302343:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3143
00000100:00080000:1.0:1787829257.302350:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1787829257.302355:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626921088:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787829257.302356:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829257.302365:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829257.302374:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626921472/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787829257.302425:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829257.302466:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829257/real 1787829257]  req@ffff8cd0d48f4b40 x1874671626921088/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829282 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829257.302481:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626921088/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829282 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829257.302490:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829257.302493:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829257.302504:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626921088:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829257.302512:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626921216:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829257.302528:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829257.302536:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829257/real 1787829257]  req@ffff8cd0d48f7480 x1874671626921216/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829282 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829257.302544:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626921216/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829282 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829257.302549:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829257.302551:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829257.302553:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626921216:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829257.302557:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626921344:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829257.302568:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829257.302573:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829257/real 1787829257]  req@ffff8cd0d48f6580 x1874671626921344/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829282 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829257.302591:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626921344/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829282 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829257.302597:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829257.302599:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829257.302601:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626921344:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829257.302620:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626921472:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829257.302634:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829257.302640:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829257/real 1787829257]  req@ffff8cd0d48f4000 x1874671626921472/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829282 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829257.302648:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626921472/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829282 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829257.302653:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829257.302654:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829257.302656:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626921472:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829257.494163:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829257.878214:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829258.518164:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829258.902320:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829259.542197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829259.926320:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829260.566282:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829260.950285:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829261.590176:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829261.974247:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787829262.422205:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829262.422213:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787829262.422216:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829262.422224:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3148
00000100:00080000:3.0:1787829262.422234:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829262.422266:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787829262.422299:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626921600/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829262.422322:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829262.422325:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787829262.422326:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829262.422329:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3148
00000100:00080000:3.0:1787829262.422332:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829262.422335:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829262.422342:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829262.422350:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626921728/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829262.422358:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829262.422362:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787829262.422364:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829262.422368:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3148
00000100:00080000:3.0:1787829262.422372:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829262.422378:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829262.422380:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626921600:192.168.203.145@tcp:250:kworker.0
00000008:00100000:3.0:1787829262.422385:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829262.422399:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626921856/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829262.422411:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829262.422414:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787829262.422421:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829262.422425:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3148
00000100:00080000:3.0:1787829262.422429:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829262.422434:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829262.422441:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1787829262.422445:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829262.422459:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626921984/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829262.422494:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829262/real 1787829262]  req@ffff8cd0c8eae1c0 x1874671626921600/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829287 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829262.422529:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626921600/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829287 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829262.422539:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829262.422543:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829262.422547:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626921600:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829262.422555:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626921728:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829262.422580:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829262.422593:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829262/real 1787829262]  req@ffff8cd0c8ead2c0 x1874671626921728/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829287 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829262.422605:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626921728/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829287 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829262.422612:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829262.422616:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829262.422619:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626921728:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829262.422644:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626921856:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829262.422664:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829262.422676:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829262/real 1787829262]  req@ffff8cd0c8eae940 x1874671626921856/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829287 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829262.422689:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626921856/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829287 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829262.422710:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829262.422728:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829262.422732:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626921856:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829262.422740:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626921984:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829262.422764:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829262.422777:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829262/real 1787829262]  req@ffff8cd0c8eaf840 x1874671626921984/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829287 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829262.422788:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626921984/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829287 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829262.422797:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829262.422800:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829262.422804:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626921984:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829262.614201:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829262.998236:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829263.638222:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829264.022203:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829264.662183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829265.046229:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829265.686259:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829266.070165:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829266.710152:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829267.094267:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829267.542352:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829267.542366:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829267.542372:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829267.542384:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3153
00000100:00080000:1.0:1787829267.542399:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829267.542407:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829267.542474:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626922112/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829267.542514:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829267.542519:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829267.542521:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829267.542526:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3153
00000100:00080000:1.0:1787829267.542531:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829267.542535:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829267.542555:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829267.542570:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626922240/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829267.542587:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1787829267.542607:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626922112:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787829267.542631:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829267.542637:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829267.542643:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3153
00000100:00080000:1.0:1787829267.542649:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829267.542654:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829267.542663:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829267.542719:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626922368/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829267.542735:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829267.542740:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829267.542743:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1787829267.542753:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787829267.542758:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3153
00000100:00080000:1.0:1787829267.542768:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829267.542774:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829267.542782:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829267.542801:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626922496/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829267.542827:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829267/real 1787829267]  req@ffff8cd0d48f7840 x1874671626922112/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829292 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829267.542855:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626922112/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829292 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829267.542869:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829267.542886:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829267.542892:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626922112:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829267.542903:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626922240:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829267.542937:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829267.542962:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829267/real 1787829267]  req@ffff8cd0d48f5a40 x1874671626922240/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829292 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829267.542987:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626922240/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829292 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829267.543003:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829267.543010:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829267.543018:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626922240:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829267.543060:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626922368:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829267.543150:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829267.543169:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829267/real 1787829267]  req@ffff8cd0d48f43c0 x1874671626922368/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829292 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829267.543186:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626922368/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829292 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829267.543197:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829267.543201:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829267.543205:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626922368:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829267.543212:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626922496:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829267.543256:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829267.543270:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829267/real 1787829267]  req@ffff8cd0d48f70c0 x1874671626922496/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829292 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829267.543289:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626922496/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829292 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829267.543305:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829267.543311:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829267.543317:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626922496:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829267.734246:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829268.118198:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829268.758170:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829269.142238:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829269.782205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829270.166206:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829270.806195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829271.190276:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829271.830192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829272.214177:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829272.662174:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829272.662183:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829272.662187:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829272.662195:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3158
00000100:00080000:2.0:1787829272.662204:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829272.662210:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829272.662239:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626922624/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829272.662285:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829272.662288:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829272.662289:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829272.662292:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3158
00000100:00080000:2.0:1787829272.662296:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829272.662299:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829272.662307:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829272.662317:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626922752/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829272.662324:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829272.662327:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829272.662329:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829272.662331:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3158
00000100:00080000:2.0:1787829272.662333:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829272.662335:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829272.662339:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829272.662342:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626922624:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787829272.662348:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626922880/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829272.662362:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829272.662365:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829272.662366:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829272.662370:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3158
00000100:00080000:2.0:1787829272.662379:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829272.662384:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829272.662389:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1787829272.662402:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787829272.662406:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626923008/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829272.662454:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829272/real 1787829272]  req@ffff8cd0c3daa940 x1874671626922624/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829297 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829272.662470:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626922624/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829297 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829272.662479:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829272.662483:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829272.662486:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626922624:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829272.662491:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626922752:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829272.662508:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829272.662525:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829272/real 1787829272]  req@ffff8cd0c3da8780 x1874671626922752/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829297 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829272.662534:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626922752/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829297 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829272.662539:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829272.662541:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829272.662544:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626922752:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829272.662562:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626922880:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829272.662575:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829272.662582:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829272/real 1787829272]  req@ffff8cd0c3da9a40 x1874671626922880/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829297 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829272.662589:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626922880/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829297 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829272.662595:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829272.662597:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829272.662599:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626922880:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829272.662602:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626923008:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829272.662612:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829272.662618:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829272/real 1787829272]  req@ffff8cd0c3dab0c0 x1874671626923008/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829297 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829272.662625:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626923008/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829297 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829272.662635:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829272.662638:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829272.662653:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626923008:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829272.854184:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829273.238277:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829273.878175:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829274.262196:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829274.902139:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829275.286283:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829275.926194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829276.310310:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829276.950167:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829277.334281:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829277.782225:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829277.782236:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829277.782241:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829277.782252:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3163
00000100:00080000:1.0:1787829277.782263:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829277.782269:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829277.782305:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626923136/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829277.782346:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829277.782351:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829277.782354:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829277.782359:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3163
00000100:00080000:1.0:1787829277.782364:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829277.782369:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829277.782381:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829277.782393:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626923136:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829277.782397:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626923264/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829277.782410:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829277.782414:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829277.782416:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829277.782420:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3163
00000100:00080000:1.0:1787829277.782424:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829277.782429:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829277.782434:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829277.782445:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626923392/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787829277.782455:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787829277.782456:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829277.782460:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829277.782462:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829277.782466:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3163
00000100:00080000:1.0:1787829277.782487:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829277.782492:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829277.782498:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829277.782500:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829277/real 1787829277]  req@ffff8cd0d48f61c0 x1874671626923136/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829302 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829277.782511:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626923520/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829277.782520:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626923136/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829302 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829277.782532:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829277.782537:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829277.782542:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626923136:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829277.782569:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626923264:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829277.782594:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829277.782625:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829277/real 1787829277]  req@ffff8cd0d48f6d00 x1874671626923264/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829302 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829277.782637:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626923264/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829302 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829277.782644:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829277.782647:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829277.782657:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626923264:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829277.782662:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626923392:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829277.782683:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829277.782693:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829277/real 1787829277]  req@ffff8cd0d48f52c0 x1874671626923392/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829302 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829277.782702:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626923392/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829302 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829277.782708:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829277.782711:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829277.782714:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626923392:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829277.782718:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626923520:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829277.782730:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829277.782737:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829277/real 1787829277]  req@ffff8cd0d48f7c00 x1874671626923520/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829302 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829277.782746:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626923520/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829302 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829277.782752:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829277.782754:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829277.782756:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626923520:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829277.974220:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829278.358279:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829278.998145:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829279.382281:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829280.022229:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829280.406230:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829281.046237:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829281.430270:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829282.070166:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829282.454243:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829282.902226:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829282.902234:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829282.902237:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829282.902245:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3168
00000100:00080000:2.0:1787829282.902253:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829282.902259:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829282.902291:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626923648/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829282.902313:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829282.902316:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829282.902317:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829282.902320:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3168
00000100:00080000:2.0:1787829282.902322:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829282.902324:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829282.902331:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829282.902339:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626923776/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829282.902358:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829282.902360:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829282.902362:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829282.902364:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3168
00000100:00080000:2.0:1787829282.902368:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829282.902371:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829282.902375:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829282.902383:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626923904/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829282.902391:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829282.902395:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:0.0:1787829282.902396:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626923648:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787829282.902397:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829282.902405:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3168
00000100:00080000:2.0:1787829282.902408:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829282.902420:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829282.902423:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829282.902433:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626924032/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787829282.902484:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829282.902536:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829282/real 1787829282]  req@ffff8cd0c3dabc00 x1874671626923648/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829307 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829282.902552:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626923648/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829307 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829282.902561:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829282.902564:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829282.902567:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626923648:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829282.902607:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626923776:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829282.902626:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829282.902635:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829282/real 1787829282]  req@ffff8cd0c3daa580 x1874671626923776/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829307 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829282.902643:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626923776/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829307 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829282.902649:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829282.902651:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829282.902653:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626923776:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829282.902688:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626923904:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829282.902702:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829282.902709:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829282/real 1787829282]  req@ffff8cd0c3dab840 x1874671626923904/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829307 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829282.902717:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626923904/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829307 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829282.902748:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829282.902750:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829282.902752:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626923904:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829282.902756:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626924032:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829282.902769:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829282.902776:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829282/real 1787829282]  req@ffff8cd0c3da8000 x1874671626924032/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829307 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829282.902783:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626924032/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829307 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829282.902788:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829282.902789:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829282.902791:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626924032:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829283.094158:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829283.478241:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829284.118151:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829284.502198:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829285.142159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829285.526302:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829286.166177:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829286.550305:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829287.190207:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829287.574211:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829288.022186:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829288.022195:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829288.022199:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829288.022207:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3173
00000100:00080000:1.0:1787829288.022217:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829288.022223:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829288.022255:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626924160/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829288.022277:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829288.022280:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829288.022282:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829288.022284:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3173
00000100:00080000:1.0:1787829288.022287:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829288.022289:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829288.022296:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829288.022305:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626924288/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829288.022313:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1787829288.022324:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626924160:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787829288.022334:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829288.022337:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829288.022341:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3173
00000100:00080000:1.0:1787829288.022344:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829288.022348:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829288.022353:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829288.022363:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626924416/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829288.022371:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829288.022373:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829288.022375:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1787829288.022380:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787829288.022380:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3173
00000100:00080000:1.0:1787829288.022387:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829288.022390:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829288.022395:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829288.022408:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626924544/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829288.022419:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829288/real 1787829288]  req@ffff8cd0d48f7c00 x1874671626924160/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829313 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829288.022433:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626924160/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829313 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829288.022440:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829288.022444:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829288.022447:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626924160:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829288.022459:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626924288:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829288.022477:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829288.022486:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829288/real 1787829288]  req@ffff8cd0d48f52c0 x1874671626924288/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829313 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829288.022494:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626924288/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829313 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829288.022499:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829288.022501:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829288.022503:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626924288:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829288.022522:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626924416:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829288.022548:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829288.022559:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829288/real 1787829288]  req@ffff8cd0d48f6d00 x1874671626924416/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829313 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829288.022567:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626924416/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829313 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829288.022572:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829288.022574:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829288.022576:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626924416:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829288.022580:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626924544:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829288.022593:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829288.022599:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829288/real 1787829288]  req@ffff8cd0d48f61c0 x1874671626924544/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829313 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829288.022613:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626924544/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829313 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829288.022619:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829288.022630:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829288.022635:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626924544:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829288.214173:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829288.598226:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829289.238216:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829289.622213:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829290.262169:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829290.646256:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829291.286197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829291.670290:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829292.310175:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829292.694228:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787829293.142200:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829293.142208:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787829293.142212:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829293.142220:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3178
00000100:00080000:0.0:1787829293.142230:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829293.142235:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829293.142267:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626924672/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829293.142300:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829293.142304:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829293.142306:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829293.142309:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3178
00000100:00080000:0.0:1787829293.142314:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829293.142318:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829293.142328:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829293.142343:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626924672:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829293.142346:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626924800/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829293.142375:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829293.142378:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787829293.142380:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829293.142383:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3178
00000100:00080000:0.0:1787829293.142387:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829293.142390:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829293.142395:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:2.0:1787829293.142404:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829293.142407:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626924928/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829293.142415:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829293.142417:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829293.142419:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829293.142422:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3178
00000100:00080000:0.0:1787829293.142426:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829293.142428:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829293.142431:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829293.142438:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626925056/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787829293.142455:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829293/real 1787829293]  req@ffff8cd0c8e9e580 x1874671626924672/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829318 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787829293.142477:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626924672/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829318 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829293.142486:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787829293.142489:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787829293.142492:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626924672:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787829293.142525:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626924800:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787829293.142544:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787829293.142554:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829293/real 1787829293]  req@ffff8cd0c8e9ed00 x1874671626924800/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829318 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787829293.142563:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626924800/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829318 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829293.142569:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787829293.142571:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787829293.142574:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626924800:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787829293.142578:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626924928:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787829293.142590:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787829293.142596:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829293/real 1787829293]  req@ffff8cd0c8e9fc00 x1874671626924928/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829318 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787829293.142616:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626924928/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829318 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829293.142621:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787829293.142624:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787829293.142626:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626924928:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787829293.142629:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626925056:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787829293.142642:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787829293.142649:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829293/real 1787829293]  req@ffff8cd0c8e9cf00 x1874671626925056/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829318 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787829293.142656:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626925056/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829318 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829293.142661:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787829293.142663:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787829293.142665:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626925056:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829293.334174:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829293.718221:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829294.358189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829294.742227:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829295.382165:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829295.766237:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829296.406200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829296.790335:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829297.430255:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829297.814263:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829298.262249:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829298.262261:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829298.262266:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829298.262278:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3184
00000100:00080000:1.0:1787829298.262293:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829298.262300:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829298.262358:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626925184/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829298.262412:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829298.262416:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829298.262417:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829298.262421:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3184
00000100:00080000:1.0:1787829298.262425:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829298.262429:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829298.262441:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829298.262454:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626925312/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829298.262478:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829298.262482:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829298.262484:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829298.262488:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3184
00000100:00080000:1.0:1787829298.262491:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829298.262496:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829298.262501:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829298.262530:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626925440/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787829298.262539:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626925184:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787829298.262544:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829298.262556:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829298.262558:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829298.262563:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3184
00000100:00080000:1.0:1787829298.262572:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829298.262576:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829298.262581:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829298.262613:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626925568/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:2.0:1787829298.262639:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787829298.262706:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829298/real 1787829298]  req@ffff8cd0d48f70c0 x1874671626925184/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829323 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787829298.262728:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626925184/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829323 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829298.262753:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787829298.262757:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787829298.262761:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626925184:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787829298.262768:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626925312:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787829298.262792:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787829298.262810:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829298/real 1787829298]  req@ffff8cd0d48f43c0 x1874671626925312/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829323 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787829298.262820:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626925312/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829323 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829298.262827:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787829298.262830:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787829298.262833:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626925312:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787829298.262856:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626925440:192.168.203.145@tcp:8:kworker.0
00000800:00000100:2.0:1787829298.262871:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787829298.262880:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829298/real 1787829298]  req@ffff8cd0d48f5a40 x1874671626925440/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829323 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787829298.262889:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626925440/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829323 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829298.262896:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787829298.262898:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787829298.262901:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626925440:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787829298.262913:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626925568:192.168.203.145@tcp:38:kworker.0
00000800:00000100:2.0:1787829298.262936:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787829298.262951:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829298/real 1787829298]  req@ffff8cd0d48f7840 x1874671626925568/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829323 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787829298.262967:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626925568/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829323 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829298.262978:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787829298.262983:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:2.0:1787829298.262988:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626925568:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829298.454141:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829298.838385:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829299.478148:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829299.862348:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829300.502140:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829300.886330:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829301.526142:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829301.910204:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829302.550134:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829302.934252:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829303.382233:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829303.382245:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829303.382249:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829303.382260:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3189
00000100:00080000:2.0:1787829303.382270:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829303.382277:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829303.382311:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626925696/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829303.382337:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829303.382340:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829303.382341:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829303.382359:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3189
00000100:00080000:2.0:1787829303.382363:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829303.382366:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829303.382375:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829303.382386:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626925696:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787829303.382388:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626925824/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829303.382402:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829303.382405:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829303.382406:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829303.382416:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3189
00000100:00080000:2.0:1787829303.382419:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829303.382422:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829303.382438:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829303.382449:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626925952/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829303.382457:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829303.382460:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829303.382463:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1787829303.382464:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787829303.382469:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3189
00000100:00080000:2.0:1787829303.382474:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829303.382479:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829303.382484:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829303.382497:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daad00 x1874671626926080/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829303.382516:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829303/real 1787829303]  req@ffff8cd0c3da9e00 x1874671626925696/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829328 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829303.382556:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626925696/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829328 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829303.382566:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829303.382569:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829303.382573:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626925696:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829303.382591:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626925824:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829303.382611:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829303.382621:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829303/real 1787829303]  req@ffff8cd0c3da92c0 x1874671626925824/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829328 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829303.382629:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626925824/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829328 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829303.382635:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829303.382637:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829303.382639:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626925824:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829303.382643:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626925952:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829303.382654:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829303.382660:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829303/real 1787829303]  req@ffff8cd0c3daa1c0 x1874671626925952/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829328 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829303.382667:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626925952/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829328 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829303.382682:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829303.382684:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829303.382686:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626925952:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829303.382691:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626926080:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829303.382712:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829303.382725:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829303/real 1787829303]  req@ffff8cd0c3daad00 x1874671626926080/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829328 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829303.382738:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daad00 x1874671626926080/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829328 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829303.382747:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829303.382750:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829303.382753:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626926080:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829303.574132:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829303.958204:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829304.598204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829304.982263:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829305.622151:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829306.006183:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829306.646217:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829307.030244:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829307.670181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829308.054255:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829308.502236:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829308.502247:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829308.502251:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829308.502261:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3194
00000100:00080000:1.0:1787829308.502272:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829308.502279:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829308.502317:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626926208/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829308.502346:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829308.502349:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829308.502352:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829308.502356:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3194
00000100:00080000:1.0:1787829308.502360:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829308.502364:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829308.502375:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829308.502388:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626926336/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829308.502404:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626926208:192.168.203.145@tcp:250:kworker.0
00000800:00000100:0.0:1787829308.502475:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787829308.502492:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829308.502509:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829308.502511:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1787829308.502515:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829308/real 1787829308]  req@ffff8cd0d48f4000 x1874671626926208/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829333 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829308.502519:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3194
00000100:00080000:1.0:1787829308.502529:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1787829308.502536:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626926208/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829333 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829308.502538:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829308.502550:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1787829308.502551:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829308.502557:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829308.502561:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626926208:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829308.502571:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626926464/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829308.502584:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829308.502588:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:0.0:1787829308.502590:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626926336:192.168.203.145@tcp:38:kworker.0
00000100:00080000:1.0:1787829308.502592:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829308.502599:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3194
00000100:00080000:1.0:1787829308.502604:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829308.502609:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:0.0:1787829308.502615:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:1.0:1787829308.502615:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829308.502628:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829308/real 1787829308]  req@ffff8cd0d48f6580 x1874671626926336/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829333 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829308.502633:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626926592/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829308.502656:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626926336/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829333 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829308.502667:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829308.502672:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829308.502677:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626926336:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829308.502685:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626926464:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829308.502704:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829308.502712:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829308/real 1787829308]  req@ffff8cd0d48f7480 x1874671626926464/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829333 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829308.502720:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626926464/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829333 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829308.502726:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829308.502728:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829308.502730:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626926464:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829308.502740:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626926592:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829308.502751:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829308.502757:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829308/real 1787829308]  req@ffff8cd0d48f4b40 x1874671626926592/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829333 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829308.502774:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626926592/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829333 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829308.502779:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829308.502781:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829308.502783:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626926592:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829308.694164:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829309.078265:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829309.718161:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829310.102313:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829310.742162:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829311.126238:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829311.766201:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829312.150247:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829312.790249:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829313.174227:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829313.622160:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829313.622170:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829313.622174:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829313.622184:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3199
00000100:00080000:1.0:1787829313.622194:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829313.622201:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829313.622237:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626926720/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829313.622280:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829313.622283:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829313.622286:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829313.622290:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3199
00000100:00080000:1.0:1787829313.622295:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829313.622301:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829313.622310:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829313.622323:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626926720:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829313.622327:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626926848/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829313.622351:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829313.622369:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829313.622371:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829313.622376:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3199
00000100:00080000:1.0:1787829313.622395:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829313.622399:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829313.622405:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829313.622416:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626926976/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829313.622426:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829313.622430:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829313.622432:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829313.622436:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3199
00000100:00080000:1.0:1787829313.622447:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829313.622453:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829313.622460:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626926848:192.168.203.145@tcp:38:kworker.0
00000008:00100000:1.0:1787829313.622460:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829313.622478:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626927104/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829313.622505:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626926976:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829313.622537:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626927104:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829313.814181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829314.198277:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829314.838201:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829315.222191:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829315.862163:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829316.246256:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:1.0:1787829316.694386:0:2344:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:1.0:1787829316.694431:0:2344:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1787829316.694448:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787829316.694486:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787829316.694500:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787829316.694511:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:0.0:1787829316.694528:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829313/real 1787829316]  req@ffff8cd0d48f7480 x1874671626926848/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829338 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829316.694553:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626926848/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829338 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829316.694564:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829316.694570:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829316.694574:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626926848:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829316.694582:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829313/real 1787829316]  req@ffff8cd0d48f6580 x1874671626926976/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829338 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829316.694595:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626926976/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829338 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829316.694618:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829316.694622:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829316.694625:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626926976:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829316.694633:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829313/real 1787829316]  req@ffff8cd0d48f4b40 x1874671626926720/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829338 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829316.694646:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626926720/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829338 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829316.694654:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829316.694658:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829316.694661:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626926720:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829316.694684:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829313/real 1787829316]  req@ffff8cd0d48f4000 x1874671626927104/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829338 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829316.694696:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626927104/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829338 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829316.694703:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829316.694707:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829316.694710:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626927104:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829316.886195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829317.270216:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829317.910150:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829318.294266:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829318.742244:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829318.742252:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829318.742256:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829318.742265:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3204
00000100:00080000:2.0:1787829318.742275:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829318.742282:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829318.742336:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daad00 x1874671626927232/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829318.742366:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829318.742370:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829318.742372:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829318.742376:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3204
00000100:00080000:2.0:1787829318.742381:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829318.742385:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829318.742393:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829318.742405:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626927360/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829318.742414:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1787829318.742422:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626927232:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787829318.742434:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829318.742437:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829318.742442:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3204
00000100:00080000:2.0:1787829318.742447:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829318.742452:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829318.742459:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829318.742473:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626927488/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829318.742484:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829318.742487:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829318.742489:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1787829318.742495:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787829318.742496:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3204
00000100:00080000:2.0:1787829318.742504:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829318.742510:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829318.742517:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829318.742531:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626927616/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829318.742611:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829318/real 1787829318]  req@ffff8cd0c3daad00 x1874671626927232/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829343 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829318.742653:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daad00 x1874671626927232/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829343 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829318.742664:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829318.742668:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829318.742672:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626927232:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829318.742697:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626927360:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829318.742725:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829318.742740:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829318/real 1787829318]  req@ffff8cd0c3daa1c0 x1874671626927360/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829343 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829318.742753:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626927360/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829343 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829318.742763:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829318.742767:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829318.742770:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626927360:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829318.742778:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626927488:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829318.742803:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829318.742816:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829318/real 1787829318]  req@ffff8cd0c3da92c0 x1874671626927488/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829343 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829318.742829:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626927488/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829343 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829318.742852:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829318.742856:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829318.742860:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626927488:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829318.742866:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626927616:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829318.742891:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829318.742905:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829318/real 1787829318]  req@ffff8cd0c3da9e00 x1874671626927616/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829343 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829318.742918:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626927616/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829343 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829318.742926:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829318.742929:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829318.742933:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626927616:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829318.934193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829319.318280:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829319.958134:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829320.342275:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829320.982180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829321.366253:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829322.006204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829322.390233:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829323.030164:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829323.414239:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787829323.862181:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829323.862191:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787829323.862196:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829323.862206:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3209
00000100:00080000:3.0:1787829323.862216:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829323.862224:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787829323.862260:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626927744/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829323.862297:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829323.862302:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787829323.862303:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829323.862308:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3209
00000100:00080000:3.0:1787829323.862312:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829323.862318:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829323.862327:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829323.862337:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626927744:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829323.862343:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626927872/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829323.862359:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829323.862362:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787829323.862364:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829323.862369:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3209
00000100:00080000:3.0:1787829323.862386:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1787829323.862393:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787829323.862394:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829323.862404:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829323.862422:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626928000/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829323.862433:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829323/real 1787829323]  req@ffff8cd0c8eacf00 x1874671626927744/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829348 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829323.862435:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829323.862446:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787829323.862451:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829323.862457:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3209
00000100:00100000:0.0:1787829323.862461:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626927744/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829348 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829323.862464:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829323.862473:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00080000:0.0:1787829323.862475:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:3.0:1787829323.862479:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1787829323.862481:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829323.862487:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626927744:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829323.862497:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626928128/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829323.862518:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626927872:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829323.862538:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829323.862547:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829323/real 1787829323]  req@ffff8cd0c8eac000 x1874671626927872/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829348 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829323.862555:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626927872/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829348 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829323.862560:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829323.862563:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829323.862565:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626927872:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829323.862568:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626928000:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829323.862579:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829323.862585:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829323/real 1787829323]  req@ffff8cd0c8eaf840 x1874671626928000/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829348 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829323.862592:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626928000/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829348 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829323.862597:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829323.862599:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829323.862600:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626928000:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829323.862604:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626928128:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829323.862613:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829323.862618:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829323/real 1787829323]  req@ffff8cd0c8eae940 x1874671626928128/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829348 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829323.862633:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626928128/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829348 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829323.862639:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829323.862640:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829323.862642:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626928128:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829324.054166:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829324.438210:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829325.078144:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829325.462241:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829326.102238:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829326.486219:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829327.126176:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829327.510208:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829328.150164:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829328.534223:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829328.982221:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829328.982237:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829328.982242:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829328.982269:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3214
00000100:00080000:1.0:1787829328.982284:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829328.982294:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829328.982347:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626928256/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829328.982380:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829328.982385:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829328.982387:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829328.982392:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3214
00000100:00080000:1.0:1787829328.982397:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829328.982420:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829328.982434:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829328.982435:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626928256:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829328.982460:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626928384/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829328.982477:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829328.982482:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829328.982485:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829328.982489:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3214
00000100:00080000:1.0:1787829328.982496:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829328.982501:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:0.0:1787829328.982506:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:1.0:1787829328.982515:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829328.982534:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626928512/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829328.982545:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829328.982554:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:0.0:1787829328.982555:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829328/real 1787829328]  req@ffff8cd0d48f7840 x1874671626928256/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829353 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829328.982561:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829328.982570:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3214
00000100:00080000:1.0:1787829328.982581:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829328.982589:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829328.982594:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626928256/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829353 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:1.0:1787829328.982598:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1787829328.982614:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829328.982620:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829328.982628:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626928640/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829328.982629:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626928256:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829328.982672:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626928384:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829328.982694:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829328.982708:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829328/real 1787829328]  req@ffff8cd0d48f5a40 x1874671626928384/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829353 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829328.982721:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626928384/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829353 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829328.982750:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829328.982754:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829328.982757:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626928384:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829328.982762:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626928512:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829328.982785:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829328.982797:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829328/real 1787829328]  req@ffff8cd0d48f43c0 x1874671626928512/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829353 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829328.982810:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626928512/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829353 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829328.982819:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829328.982823:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829328.982826:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626928512:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829328.982830:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626928640:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829328.982847:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829328.982855:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829328/real 1787829328]  req@ffff8cd0d48f70c0 x1874671626928640/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829353 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829328.982862:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626928640/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829353 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829328.982867:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829328.982870:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829328.982879:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626928640:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829329.174164:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829329.558342:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829330.198183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829330.582210:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829331.222166:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829331.606286:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829332.246166:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829332.630259:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829333.270161:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829333.654251:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829334.102223:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829334.102232:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829334.102236:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829334.102244:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3219
00000100:00080000:2.0:1787829334.102254:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829334.102260:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829334.102292:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626928768/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829334.102315:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829334.102318:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829334.102320:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829334.102324:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3219
00000100:00080000:2.0:1787829334.102328:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829334.102332:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829334.102341:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829334.102356:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626928896/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829334.102362:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626928768:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787829334.102367:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829334.102371:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829334.102373:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829334.102376:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3219
00000100:00080000:2.0:1787829334.102380:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829334.102383:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829334.102388:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1787829334.102420:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787829334.102424:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626929024/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829334.102433:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829334.102436:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829334.102438:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829334.102442:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3219
00000100:00080000:2.0:1787829334.102445:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829334.102450:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829334.102455:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829334.102465:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626929152/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829334.102467:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829334/real 1787829334]  req@ffff8cd0c3da8000 x1874671626928768/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829359 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829334.102487:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626928768/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829359 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829334.102497:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829334.102501:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829334.102505:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626928768:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829334.102527:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626928896:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829334.102547:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829334.102560:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829334/real 1787829334]  req@ffff8cd0c3dab840 x1874671626928896/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829359 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829334.102572:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626928896/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829359 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829334.102581:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829334.102585:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829334.102588:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626928896:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829334.102595:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626929024:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829334.102616:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829334.102633:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829334/real 1787829334]  req@ffff8cd0c3daa580 x1874671626929024/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829359 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829334.102641:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626929024/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829359 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829334.102647:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829334.102649:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829334.102651:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626929024:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829334.102655:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626929152:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829334.102667:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829334.102674:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829334/real 1787829334]  req@ffff8cd0c3dabc00 x1874671626929152/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829359 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829334.102681:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626929152/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829359 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829334.102686:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829334.102687:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829334.102689:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626929152:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829334.294171:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829334.678199:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829335.318153:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829335.702270:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829336.342183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829336.726210:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829337.366155:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829337.750236:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829338.390181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829338.774232:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829339.222223:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829339.222246:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829339.222249:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829339.222258:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3225
00000100:00080000:1.0:1787829339.222266:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829339.222271:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829339.222299:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626929280/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829339.222327:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829339.222329:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829339.222331:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829339.222333:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3225
00000100:00080000:1.0:1787829339.222336:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829339.222338:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829339.222345:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829339.222354:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626929408/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829339.222362:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829339.222365:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829339.222366:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829339.222368:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3225
00000100:00080000:1.0:1787829339.222371:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1787829339.222376:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626929280:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787829339.222379:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829339.222389:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829339.222402:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626929536/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829339.222422:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829339.222425:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829339.222427:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829339.222431:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3225
00000100:00080000:1.0:1787829339.222435:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829339.222438:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829339.222443:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1787829339.222446:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829339.222455:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626929664/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829339.222489:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829339/real 1787829339]  req@ffff8cd0d48f61c0 x1874671626929280/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829364 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829339.222503:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626929280/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829364 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829339.222517:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829339.222522:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829339.222526:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626929280:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829339.222534:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626929408:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829339.222587:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829339.222600:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829339/real 1787829339]  req@ffff8cd0d48f6d00 x1874671626929408/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829364 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829339.222609:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626929408/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829364 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829339.222615:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829339.222618:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829339.222620:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626929408:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829339.222662:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626929536:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829339.222684:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829339.222692:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829339/real 1787829339]  req@ffff8cd0d48f52c0 x1874671626929536/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829364 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829339.222700:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626929536/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829364 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829339.222706:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829339.222708:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829339.222710:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626929536:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829339.222737:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626929664:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829339.222752:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829339.222760:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829339/real 1787829339]  req@ffff8cd0d48f7c00 x1874671626929664/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829364 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829339.222768:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626929664/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829364 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829339.222773:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829339.222775:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829339.222777:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626929664:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829339.414139:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829339.798261:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829340.438187:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829340.822235:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829341.462136:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829341.846267:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829342.486210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829342.870225:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829343.510260:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829343.894253:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787829344.342242:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829344.342253:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787829344.342258:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829344.342269:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3230
00000100:00080000:3.0:1787829344.342282:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829344.342289:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787829344.342331:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626929792/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829344.342358:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829344.342361:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787829344.342362:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829344.342388:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3230
00000100:00080000:3.0:1787829344.342393:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829344.342399:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829344.342414:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829344.342415:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626929792:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829344.342436:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626929920/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829344.342451:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829344.342455:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787829344.342456:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829344.342460:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3230
00000100:00080000:3.0:1787829344.342465:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829344.342469:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829344.342473:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829344.342484:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626930048/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829344.342493:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:0.0:1787829344.342495:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787829344.342500:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787829344.342503:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829344.342509:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3230
00000100:00080000:3.0:1787829344.342515:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829344.342520:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829344.342526:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829344.342547:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626930176/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829344.342552:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829344/real 1787829344]  req@ffff8cd0c8ead2c0 x1874671626929792/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829369 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829344.342584:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626929792/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829369 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829344.342598:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829344.342603:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829344.342608:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626929792:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829344.342638:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626929920:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829344.342659:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829344.342670:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829344/real 1787829344]  req@ffff8cd0c8eae1c0 x1874671626929920/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829369 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829344.342687:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626929920/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829369 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829344.342693:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829344.342696:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829344.342698:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626929920:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829344.342702:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626930048:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829344.342727:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829344.342736:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829344/real 1787829344]  req@ffff8cd0c8eada40 x1874671626930048/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829369 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829344.342744:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626930048/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829369 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829344.342750:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829344.342752:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829344.342754:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626930048:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829344.342758:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626930176:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829344.342776:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829344.342783:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829344/real 1787829344]  req@ffff8cd0c8eac780 x1874671626930176/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829369 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829344.342791:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626930176/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829369 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829344.342806:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829344.342808:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829344.342810:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626930176:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829344.534216:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829344.918253:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829345.558213:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829345.942308:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829346.582191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829346.966212:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829347.606215:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829347.990227:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829348.630180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829349.014245:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787829349.462151:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829349.462159:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787829349.462163:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829349.462171:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3235
00000100:00080000:3.0:1787829349.462181:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829349.462186:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787829349.462216:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626930304/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829349.462239:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829349.462241:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787829349.462243:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829349.462245:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3235
00000100:00080000:3.0:1787829349.462247:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829349.462249:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829349.462258:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829349.462267:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626930432/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829349.462279:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626930304:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787829349.462296:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829349.462298:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787829349.462300:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829349.462304:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3235
00000100:00080000:3.0:1787829349.462307:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829349.462310:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829349.462314:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829349.462324:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626930560/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829349.462331:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829349.462333:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787829349.462334:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829349.462337:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3235
00000100:00080000:3.0:1787829349.462339:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829349.462341:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829349.462344:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1787829349.462351:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829349.462353:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626930688/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829349.462401:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829349/real 1787829349]  req@ffff8cd0c8eaf0c0 x1874671626930304/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829374 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829349.462416:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626930304/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829374 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829349.462424:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829349.462427:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829349.462429:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626930304:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829349.462452:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626930432:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829349.462470:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829349.462479:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829349/real 1787829349]  req@ffff8cd0c8eac3c0 x1874671626930432/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829374 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829349.462487:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626930432/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829374 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829349.462493:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829349.462495:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829349.462498:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626930432:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829349.462510:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626930560:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829349.462526:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829349.462533:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829349/real 1787829349]  req@ffff8cd0c8eac780 x1874671626930560/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829374 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829349.462541:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626930560/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829374 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829349.462546:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829349.462548:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829349.462551:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626930560:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829349.462554:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626930688:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829349.462564:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829349.462570:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829349/real 1787829349]  req@ffff8cd0c8eada40 x1874671626930688/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829374 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829349.462577:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626930688/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829374 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829349.462582:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829349.462583:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829349.462585:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626930688:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829349.654190:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829350.038247:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829350.678196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829351.062235:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829351.702155:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829352.086222:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829352.726172:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829353.110276:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829353.750175:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829354.134204:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829354.582166:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829354.582174:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829354.582177:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829354.582185:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3240
00000100:00080000:2.0:1787829354.582194:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829354.582213:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829354.582241:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626930816/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829354.582268:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829354.582271:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829354.582273:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829354.582275:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3240
00000100:00080000:2.0:1787829354.582279:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829354.582282:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829354.582290:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829354.582298:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626930944/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829354.582307:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829354.582311:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:0.0:1787829354.582314:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626930816:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787829354.582314:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829354.582322:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3240
00000100:00080000:2.0:1787829354.582324:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829354.582327:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829354.582330:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829354.582340:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626931072/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829354.582347:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829354.582349:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829354.582355:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829354.582359:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3240
00000100:00080000:2.0:1787829354.582362:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829354.582365:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829354.582369:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1787829354.582382:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787829354.582398:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626931200/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829354.582436:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829354/real 1787829354]  req@ffff8cd0c3dab0c0 x1874671626930816/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829379 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829354.582458:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626930816/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829379 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829354.582467:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829354.582471:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829354.582474:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626930816:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829354.582494:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626930944:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829354.582514:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829354.582524:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829354/real 1787829354]  req@ffff8cd0c3da9a40 x1874671626930944/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829379 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829354.582535:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626930944/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829379 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829354.582544:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829354.582548:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829354.582552:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626930944:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829354.582558:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626931072:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829354.582581:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829354.582592:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829354/real 1787829354]  req@ffff8cd0c3da8780 x1874671626931072/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829379 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829354.582601:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626931072/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829379 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829354.582616:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829354.582619:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829354.582621:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626931072:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829354.582626:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626931200:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829354.582642:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829354.582649:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829354/real 1787829354]  req@ffff8cd0c3daa940 x1874671626931200/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829379 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829354.582656:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626931200/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829379 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829354.582662:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829354.582663:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829354.582665:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626931200:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829354.774164:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829355.158182:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829355.798209:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829356.182201:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829356.822158:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829357.206225:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829357.846197:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829358.230239:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829358.870136:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829359.254248:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829359.702164:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829359.702176:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829359.702180:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829359.702191:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3245
00000100:00080000:2.0:1787829359.702202:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829359.702210:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829359.702249:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626931328/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829359.702283:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829359.702288:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829359.702291:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829359.702296:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3245
00000100:00080000:2.0:1787829359.702300:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829359.702305:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829359.702316:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829359.702337:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626931328:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787829359.702352:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8b40 x1874671626931456/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829359.702384:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829359.702388:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829359.702391:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829359.702395:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3245
00000100:00080000:2.0:1787829359.702400:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829359.702405:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:0.0:1787829359.702411:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:2.0:1787829359.702415:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829359.702446:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626931584/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829359.702459:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829359.702463:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829359.702467:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1787829359.702468:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829359/real 1787829359]  req@ffff8cd0c3da8f00 x1874671626931328/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829384 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829359.702479:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3245
00000100:00080000:2.0:1787829359.702484:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829359.702489:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829359.702497:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829359.702501:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626931328/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829384 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829359.702512:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:2.0:1787829359.702513:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626931712/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829359.702518:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829359.702531:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626931328:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829359.702553:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626931456:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829359.702590:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829359.702600:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829359/real 1787829359]  req@ffff8cd0c3da8b40 x1874671626931456/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829384 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829359.702615:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8b40 x1874671626931456/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829384 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829359.702621:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829359.702624:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829359.702629:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626931456:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829359.702649:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626931584:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829359.702664:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829359.702672:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829359/real 1787829359]  req@ffff8cd0c3da83c0 x1874671626931584/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829384 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829359.702679:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626931584/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829384 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829359.702685:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829359.702687:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829359.702689:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626931584:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829359.702693:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626931712:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829359.702704:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829359.702723:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829359/real 1787829359]  req@ffff8cd0c3da9680 x1874671626931712/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829384 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829359.702737:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626931712/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829384 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829359.702745:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829359.702748:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829359.702752:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626931712:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829359.894194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829360.278234:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829360.918188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829361.302214:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829361.942174:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829362.326248:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829362.966190:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829363.350218:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829363.990151:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829364.374213:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829364.822153:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829364.822176:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829364.822179:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829364.822188:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3250
00000100:00080000:2.0:1787829364.822197:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829364.822202:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829364.822232:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626931840/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829364.822256:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829364.822259:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829364.822260:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829364.822263:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3250
00000100:00080000:2.0:1787829364.822266:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829364.822268:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829364.822275:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829364.822284:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626931968/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829364.822292:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829364.822295:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829364.822296:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829364.822298:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3250
00000100:00080000:2.0:1787829364.822300:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829364.822305:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829364.822307:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626931840:192.168.203.145@tcp:250:kworker.0
00000008:00100000:2.0:1787829364.822309:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829364.822320:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8b40 x1874671626932096/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829364.822333:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829364.822336:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829364.822337:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829364.822340:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3250
00000100:00080000:2.0:1787829364.822344:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829364.822347:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829364.822350:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829364.822360:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626932224/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787829364.822365:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829364.822412:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829364/real 1787829364]  req@ffff8cd0c3da9680 x1874671626931840/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829364.822426:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626931840/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829364.822434:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829364.822438:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829364.822441:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626931840:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829364.822448:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626931968:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829364.822463:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829364.822471:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829364/real 1787829364]  req@ffff8cd0c3da83c0 x1874671626931968/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829364.822490:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626931968/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829364.822495:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829364.822497:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829364.822500:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626931968:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829364.822519:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626932096:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829364.822540:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829364.822553:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829364/real 1787829364]  req@ffff8cd0c3da8b40 x1874671626932096/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829364.822565:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8b40 x1874671626932096/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829364.822574:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829364.822577:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829364.822579:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626932096:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829364.822586:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626932224:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829364.822608:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829364.822620:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829364/real 1787829364]  req@ffff8cd0c3da8f00 x1874671626932224/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829364.822643:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626932224/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829364.822660:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829364.822663:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829364.822666:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626932224:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829365.014143:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829365.398239:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829366.038150:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829366.422213:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829367.062143:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829367.446192:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829368.086153:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829368.470233:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829369.110159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829369.494185:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829369.942209:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829369.942219:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829369.942224:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829369.942244:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3255
00000100:00080000:2.0:1787829369.942255:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829369.942263:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829369.942303:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626932352/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829369.942334:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829369.942338:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829369.942340:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829369.942344:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3255
00000100:00080000:2.0:1787829369.942362:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829369.942368:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829369.942379:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829369.942392:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626932352:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787829369.942396:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626932480/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829369.942412:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829369.942415:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829369.942417:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829369.942421:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3255
00000100:00080000:2.0:1787829369.942426:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829369.942431:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829369.942436:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829369.942449:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626932608/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829369.942459:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829369.942463:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829369.942464:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829369.942469:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3255
00000800:00000100:0.0:1787829369.942475:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787829369.942476:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829369.942483:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829369.942489:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829369.942499:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626932736/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829369.942523:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829369/real 1787829369]  req@ffff8cd0c3daa940 x1874671626932352/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829369.942544:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626932352/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829369.942556:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829369.942561:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829369.942566:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626932352:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829369.942590:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626932480:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829369.942630:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829369.942645:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829369/real 1787829369]  req@ffff8cd0c3da8780 x1874671626932480/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829369.942658:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626932480/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829369.942680:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829369.942683:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829369.942687:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626932480:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829369.942694:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626932608:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829369.942718:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829369.942730:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829369/real 1787829369]  req@ffff8cd0c3da9a40 x1874671626932608/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829369.942743:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626932608/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829369.942751:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829369.942754:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829369.942757:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626932608:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829369.942763:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626932736:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829369.942780:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829369.942790:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829369/real 1787829369]  req@ffff8cd0c3dab0c0 x1874671626932736/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829369.942802:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626932736/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829369.942810:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829369.942814:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829369.942824:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626932736:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829370.134143:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829370.518280:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829371.158159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829371.542185:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829372.182178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829372.566205:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829373.206172:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829373.590186:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829374.230164:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829374.614339:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829375.062179:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829375.062192:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829375.062196:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829375.062203:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3260
00000100:00080000:1.0:1787829375.062212:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829375.062218:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829375.062251:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626932864/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829375.062270:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829375.062273:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829375.062274:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829375.062277:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3260
00000100:00080000:1.0:1787829375.062279:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829375.062281:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829375.062300:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829375.062311:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626932992/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829375.062320:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829375.062323:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829375.062324:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829375.062329:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3260
00000100:00080000:1.0:1787829375.062336:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1787829375.062337:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626932864:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787829375.062341:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829375.062356:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829375.062365:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626933120/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829375.062372:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829375.062375:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829375.062376:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829375.062379:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3260
00000100:00080000:1.0:1787829375.062381:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829375.062384:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829375.062387:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829375.062395:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626933248/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787829375.062432:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829375.062496:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829375/real 1787829375]  req@ffff8cd0d48f7c00 x1874671626932864/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829400 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829375.062517:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626932864/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829400 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829375.062527:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829375.062531:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829375.062534:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626932864:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829375.062560:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626932992:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829375.062580:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829375.062590:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829375/real 1787829375]  req@ffff8cd0d48f52c0 x1874671626932992/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829400 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829375.062611:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626932992/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829400 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829375.062618:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829375.062621:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829375.062623:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626932992:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829375.062628:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626933120:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829375.062658:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829375.062668:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829375/real 1787829375]  req@ffff8cd0d48f6d00 x1874671626933120/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829400 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829375.062677:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626933120/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829400 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829375.062683:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829375.062685:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829375.062688:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626933120:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829375.062692:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626933248:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829375.062703:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829375.062709:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829375/real 1787829375]  req@ffff8cd0d48f61c0 x1874671626933248/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829400 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829375.062717:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626933248/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829400 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829375.062723:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829375.062725:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829375.062729:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626933248:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829375.254177:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829375.638234:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829376.278163:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829376.662279:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829377.302178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829377.686194:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829378.326145:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829378.710263:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829379.350147:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829379.734215:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787829380.182218:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829380.182227:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787829380.182246:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829380.182254:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3266
00000100:00080000:0.0:1787829380.182264:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829380.182270:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829380.182301:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626933376/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829380.182324:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829380.182327:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829380.182328:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829380.182331:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3266
00000100:00080000:0.0:1787829380.182333:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829380.182335:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829380.182343:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829380.182351:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626933504/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829380.182360:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829380.182365:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:2.0:1787829380.182366:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626933376:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787829380.182367:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829380.182375:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3266
00000100:00080000:0.0:1787829380.182378:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829380.182380:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829380.182383:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829380.182392:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626933632/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829380.182409:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829380.182412:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829380.182413:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829380.182417:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3266
00000100:00080000:0.0:1787829380.182420:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829380.182423:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829380.182427:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829380.182435:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626933760/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:2.0:1787829380.182467:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626933504:192.168.203.145@tcp:38:kworker.0
00000100:00100000:2.0:1787829380.182489:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626933632:192.168.203.145@tcp:8:kworker.0
00000100:00100000:2.0:1787829380.182505:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626933760:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829380.374151:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829380.758248:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829381.398185:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829381.782230:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829382.422192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829382.806258:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:2.0:1787829383.254381:0:2346:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:2.0:1787829383.254430:0:2346:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:2.0:1787829383.254447:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787829383.254486:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787829383.254496:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787829383.254505:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:0.0:1787829383.254538:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829380/real 1787829383]  req@ffff8cd0c8e9d680 x1874671626933504/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829405 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829383.254565:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626933504/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829405 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829383.254589:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829383.254593:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829383.254598:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626933504:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829383.254607:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829380/real 1787829383]  req@ffff8cd0c8e9c3c0 x1874671626933632/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829405 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829383.254621:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626933632/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829405 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829383.254651:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829383.254655:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829383.254659:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626933632:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829383.254667:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829380/real 1787829383]  req@ffff8cd0c8e9e940 x1874671626933760/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829405 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829383.254681:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626933760/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829405 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829383.254689:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829383.254693:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829383.254696:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626933760:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829383.254702:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829380/real 1787829383]  req@ffff8cd0c8e9de00 x1874671626933376/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829405 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829383.254714:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626933376/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829405 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829383.254721:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829383.254724:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829383.254728:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626933376:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829383.446162:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829383.830215:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829384.470222:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829384.854280:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829385.302203:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829385.302220:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829385.302224:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829385.302234:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3271
00000100:00080000:2.0:1787829385.302245:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829385.302250:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829385.302297:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626933888/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829385.302322:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829385.302326:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829385.302328:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829385.302332:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3271
00000100:00080000:2.0:1787829385.302337:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829385.302342:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829385.302352:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829385.302365:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626933888:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787829385.302366:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626934016/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829385.302377:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829385.302380:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829385.302382:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829385.302385:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3271
00000100:00080000:2.0:1787829385.302388:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829385.302391:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829385.302396:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829385.302405:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626934144/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829385.302414:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829385.302417:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829385.302419:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1787829385.302420:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787829385.302428:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3271
00000100:00080000:2.0:1787829385.302432:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829385.302435:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829385.302440:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829385.302450:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626934272/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829385.302460:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829385/real 1787829385]  req@ffff8cd0c3dabc00 x1874671626933888/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829410 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829385.302483:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626933888/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829410 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829385.302494:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829385.302498:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829385.302502:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626933888:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829385.302525:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626934016:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829385.302552:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829385.302566:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829385/real 1787829385]  req@ffff8cd0c3daa580 x1874671626934016/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829410 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829385.302579:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626934016/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829410 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829385.302588:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829385.302592:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829385.302595:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626934016:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829385.302601:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626934144:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829385.302624:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829385.302637:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829385/real 1787829385]  req@ffff8cd0c3dab840 x1874671626934144/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829410 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829385.302651:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626934144/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829410 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829385.302673:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829385.302677:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829385.302681:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626934144:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829385.302687:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626934272:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829385.302712:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829385.302723:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829385/real 1787829385]  req@ffff8cd0c3da8000 x1874671626934272/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829410 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829385.302731:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626934272/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829410 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829385.302738:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829385.302742:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829385.302745:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626934272:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829385.494190:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829385.878227:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829386.518205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829386.902273:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829387.542168:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829387.926211:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829388.566186:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829388.950331:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829389.590158:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829389.974231:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787829390.422261:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829390.422269:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787829390.422273:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829390.422281:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3276
00000100:00080000:3.0:1787829390.422290:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829390.422295:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787829390.422328:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626934400/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829390.422354:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829390.422357:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787829390.422358:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829390.422361:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3276
00000100:00080000:3.0:1787829390.422363:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829390.422365:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829390.422372:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829390.422380:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626934528/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829390.422389:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829390.422391:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:0.0:1787829390.422404:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626934400:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787829390.422407:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829390.422414:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3276
00000100:00080000:3.0:1787829390.422416:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829390.422420:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829390.422424:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829390.422434:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626934656/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829390.422446:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829390.422449:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787829390.422451:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829390.422455:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3276
00000100:00080000:3.0:1787829390.422459:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829390.422464:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829390.422469:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1787829390.422473:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829390.422486:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626934784/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829390.422518:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829390/real 1787829390]  req@ffff8cd0c8eae1c0 x1874671626934400/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829415 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829390.422533:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626934400/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829415 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829390.422551:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829390.422554:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829390.422557:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626934400:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829390.422568:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626934528:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829390.422587:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829390.422597:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829390/real 1787829390]  req@ffff8cd0c8ead2c0 x1874671626934528/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829415 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829390.422605:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626934528/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829415 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829390.422610:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829390.422612:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829390.422614:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626934528:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829390.422633:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626934656:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829390.422645:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829390.422651:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829390/real 1787829390]  req@ffff8cd0c8eae940 x1874671626934656/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829415 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829390.422659:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626934656/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829415 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829390.422664:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829390.422666:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829390.422668:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626934656:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829390.422677:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626934784:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829390.422691:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829390.422697:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829390/real 1787829390]  req@ffff8cd0c8eaf840 x1874671626934784/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829415 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829390.422705:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626934784/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829415 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829390.422710:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829390.422712:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829390.422714:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626934784:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829390.614183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829390.998218:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829391.638188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829392.022213:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829392.662218:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829393.046280:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829393.686149:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829394.070244:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829394.710171:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829395.094204:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829395.542244:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829395.542258:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829395.542264:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829395.542277:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3281
00000100:00080000:2.0:1787829395.542292:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829395.542301:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829395.542352:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626934912/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829395.542388:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829395.542393:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829395.542395:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829395.542399:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3281
00000100:00080000:2.0:1787829395.542404:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829395.542409:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829395.542441:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626934912:192.168.203.145@tcp:250:kworker.0
00000008:00100000:2.0:1787829395.542442:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829395.542463:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626935040/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829395.542477:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829395.542482:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829395.542484:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829395.542489:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3281
00000100:00080000:2.0:1787829395.542494:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829395.542499:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829395.542505:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1787829395.542519:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787829395.542520:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626935168/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829395.542534:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829395.542538:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829395.542540:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829395.542545:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3281
00000100:00080000:2.0:1787829395.542549:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829395.542554:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829395.542560:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829395.542572:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daad00 x1874671626935296/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829395.542579:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829395/real 1787829395]  req@ffff8cd0c3da9e00 x1874671626934912/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829420 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829395.542601:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626934912/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829420 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829395.542611:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829395.542615:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829395.542619:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626934912:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829395.542642:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626935040:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829395.542670:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829395.542680:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829395/real 1787829395]  req@ffff8cd0c3da92c0 x1874671626935040/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829420 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829395.542689:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626935040/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829420 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829395.542695:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829395.542697:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829395.542699:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626935040:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829395.542703:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626935168:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829395.542715:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829395.542721:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829395/real 1787829395]  req@ffff8cd0c3daa1c0 x1874671626935168/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829420 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829395.542728:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626935168/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829420 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829395.542733:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829395.542735:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829395.542737:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626935168:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829395.542740:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626935296:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829395.542751:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829395.542756:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829395/real 1787829395]  req@ffff8cd0c3daad00 x1874671626935296/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829420 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829395.542774:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daad00 x1874671626935296/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829420 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829395.542780:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829395.542782:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829395.542784:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626935296:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829395.734192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829396.118253:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829396.758204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829397.142245:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829397.782181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829398.166202:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829398.806180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829399.190256:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829399.830131:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829400.214167:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829400.662156:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829400.662166:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829400.662171:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829400.662181:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3286
00000100:00080000:2.0:1787829400.662192:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829400.662199:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829400.662234:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daad00 x1874671626935424/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829400.662265:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829400.662269:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829400.662271:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829400.662276:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3286
00000100:00080000:2.0:1787829400.662280:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829400.662285:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829400.662294:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829400.662299:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626935424:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787829400.662308:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626935552/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829400.662319:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829400.662323:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829400.662324:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829400.662328:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3286
00000100:00080000:2.0:1787829400.662331:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829400.662335:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829400.662346:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829400.662359:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626935680/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1787829400.662368:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787829400.662369:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829400.662372:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829400.662374:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829400.662377:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3286
00000100:00080000:2.0:1787829400.662380:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829400.662384:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829400.662388:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829400.662397:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626935808/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1787829400.662403:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829400/real 1787829400]  req@ffff8cd0c3daad00 x1874671626935424/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829425 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829400.662420:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daad00 x1874671626935424/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829425 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829400.662431:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829400.662436:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829400.662441:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626935424:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829400.662464:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626935552:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829400.662485:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829400.662497:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829400/real 1787829400]  req@ffff8cd0c3daa1c0 x1874671626935552/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829425 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829400.662506:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626935552/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829425 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829400.662512:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829400.662514:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829400.662516:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626935552:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787829400.662520:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626935680:192.168.203.145@tcp:8:kworker.0
00000800:00000100:1.0:1787829400.662531:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829400.662539:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829400/real 1787829400]  req@ffff8cd0c3da92c0 x1874671626935680/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829425 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829400.662568:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626935680/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829425 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829400.662577:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829400.662580:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829400.662584:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626935680:192.168.203.145@tcp:8:kworker.0
00000100:00100000:1.0:1787829400.662591:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626935808:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829400.662610:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829400.662618:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829400/real 1787829400]  req@ffff8cd0c3da9e00 x1874671626935808/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829425 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829400.662626:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626935808/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829425 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829400.662631:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829400.662633:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829400.662636:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626935808:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829400.854150:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829401.238163:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829401.878210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829402.262300:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829402.902171:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829403.286225:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829403.926193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829404.310249:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829404.950168:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829405.334208:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829405.782177:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829405.782186:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829405.782190:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829405.782197:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3291
00000100:00080000:2.0:1787829405.782207:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829405.782213:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829405.782252:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626935936/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829405.782287:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829405.782290:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829405.782291:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829405.782294:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3291
00000100:00080000:2.0:1787829405.782297:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829405.782300:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829405.782308:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829405.782319:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626935936:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787829405.782320:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626936064/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829405.782334:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829405.782336:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829405.782337:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829405.782340:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3291
00000100:00080000:2.0:1787829405.782343:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829405.782347:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829405.782352:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829405.782365:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626936192/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829405.782376:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000800:00000100:1.0:1787829405.782380:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787829405.782384:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829405.782388:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829405.782394:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3291
00000100:00080000:2.0:1787829405.782412:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829405.782416:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829405.782421:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829405.782423:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829405/real 1787829405]  req@ffff8cd0c3da8000 x1874671626935936/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829430 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787829405.782434:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626936320/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1787829405.782447:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626935936/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829430 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829405.782470:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829405.782474:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829405.782478:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626935936:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829405.782497:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626936064:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829405.782516:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829405.782526:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829405/real 1787829405]  req@ffff8cd0c3dab840 x1874671626936064/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829430 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829405.782534:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626936064/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829430 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829405.782540:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829405.782543:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829405.782545:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626936064:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787829405.782549:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626936192:192.168.203.145@tcp:8:kworker.0
00000800:00000100:1.0:1787829405.782560:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829405.782566:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829405/real 1787829405]  req@ffff8cd0c3daa580 x1874671626936192/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829430 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829405.782574:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626936192/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829430 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829405.782579:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829405.782581:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829405.782583:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626936192:192.168.203.145@tcp:8:kworker.0
00000100:00100000:1.0:1787829405.782592:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626936320:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829405.782605:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829405.782612:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829405/real 1787829405]  req@ffff8cd0c3dabc00 x1874671626936320/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829430 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829405.782619:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626936320/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829430 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829405.782624:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829405.782626:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829405.782628:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626936320:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829405.974189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829406.358271:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829406.998199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829407.382239:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829408.022238:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829408.406264:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829409.046163:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829409.430255:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829410.070161:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829410.454225:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829410.902163:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829410.902171:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829410.902174:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829410.902182:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3296
00000100:00080000:1.0:1787829410.902191:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829410.902196:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829410.902238:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626936448/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829410.902265:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829410.902268:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829410.902270:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829410.902272:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3296
00000100:00080000:1.0:1787829410.902275:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829410.902293:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829410.902301:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829410.902308:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626936448:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829410.902313:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626936576/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829410.902323:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829410.902325:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829410.902327:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829410.902330:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3296
00000100:00080000:1.0:1787829410.902333:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829410.902335:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829410.902339:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829410.902347:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626936704/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829410.902353:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829410.902356:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829410.902358:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829410.902360:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3296
00000100:00080000:1.0:1787829410.902362:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829410.902364:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:0.0:1787829410.902369:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:1.0:1787829410.902370:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829410.902379:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626936832/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829410.902410:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829410/real 1787829410]  req@ffff8cd0d48f70c0 x1874671626936448/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829435 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829410.902424:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626936448/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829435 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829410.902432:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829410.902435:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829410.902438:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626936448:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829410.902455:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626936576:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829410.902470:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829410.902478:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829410/real 1787829410]  req@ffff8cd0d48f43c0 x1874671626936576/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829435 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829410.902486:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626936576/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829435 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829410.902500:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829410.902502:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829410.902504:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626936576:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829410.902508:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626936704:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829410.902523:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829410.902530:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829410/real 1787829410]  req@ffff8cd0d48f5a40 x1874671626936704/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829435 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829410.902537:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626936704/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829435 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829410.902543:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829410.902544:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829410.902546:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626936704:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829410.902550:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626936832:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829410.902560:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829410.902566:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829410/real 1787829410]  req@ffff8cd0d48f7840 x1874671626936832/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829435 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829410.902572:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626936832/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829435 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829410.902577:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829410.902579:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829410.902586:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626936832:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829411.094206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829411.478258:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829412.118185:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829412.502235:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829413.142154:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829413.526262:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829414.166182:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829414.550189:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829415.190161:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829415.574211:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829416.022215:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829416.022225:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829416.022229:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829416.022239:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3301
00000100:00080000:2.0:1787829416.022250:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829416.022258:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829416.022297:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626936960/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829416.022328:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829416.022332:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829416.022334:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829416.022352:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3301
00000100:00080000:2.0:1787829416.022357:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829416.022361:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829416.022369:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829416.022370:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626936960:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787829416.022383:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626937088/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829416.022393:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829416.022396:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829416.022397:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829416.022400:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3301
00000100:00080000:2.0:1787829416.022402:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829416.022404:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829416.022409:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829416.022416:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626937216/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829416.022423:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829416.022426:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829416.022427:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829416.022429:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3301
00000100:00080000:2.0:1787829416.022432:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829416.022436:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:0.0:1787829416.022438:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:2.0:1787829416.022440:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829416.022448:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626937344/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829416.022483:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829416/real 1787829416]  req@ffff8cd0c3dab0c0 x1874671626936960/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829441 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829416.022497:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626936960/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829441 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829416.022505:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829416.022508:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829416.022511:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626936960:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829416.022529:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626937088:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829416.022547:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829416.022556:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829416/real 1787829416]  req@ffff8cd0c3da9a40 x1874671626937088/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829441 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829416.022568:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626937088/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829441 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829416.022578:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829416.022581:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829416.022585:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626937088:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829416.022592:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626937216:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829416.022614:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829416.022634:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829416/real 1787829416]  req@ffff8cd0c3da8780 x1874671626937216/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829441 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829416.022655:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626937216/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829441 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829416.022662:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829416.022664:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829416.022667:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626937216:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829416.022671:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626937344:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829416.022688:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829416.022696:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829416/real 1787829416]  req@ffff8cd0c3daa940 x1874671626937344/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829441 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829416.022703:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626937344/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829441 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829416.022709:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829416.022711:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829416.022713:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626937344:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829416.214216:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829416.598314:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829417.238154:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829417.622285:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829418.262182:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829418.646217:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829419.286141:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829419.670210:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829420.310153:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829420.694205:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829421.142166:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829421.142187:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829421.142191:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829421.142199:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3306
00000100:00080000:1.0:1787829421.142207:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829421.142213:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829421.142243:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626937472/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829421.142266:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829421.142269:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829421.142270:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829421.142272:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3306
00000100:00080000:1.0:1787829421.142275:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829421.142277:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829421.142285:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829421.142293:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626937600/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829421.142301:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829421.142304:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829421.142305:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829421.142308:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3306
00000100:00080000:1.0:1787829421.142310:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829421.142312:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829421.142316:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829421.142327:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626937472:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829421.142332:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626937728/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829421.142347:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829421.142349:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829421.142351:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829421.142354:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3306
00000100:00080000:1.0:1787829421.142358:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829421.142361:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829421.142365:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829421.142374:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626937856/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787829421.142389:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829421.142434:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829421/real 1787829421]  req@ffff8cd0d48f4000 x1874671626937472/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829446 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829421.142451:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626937472/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829446 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829421.142469:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829421.142474:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829421.142478:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626937472:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829421.142486:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626937600:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829421.142510:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829421.142522:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829421/real 1787829421]  req@ffff8cd0d48f4b40 x1874671626937600/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829446 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829421.142534:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626937600/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829446 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829421.142542:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829421.142545:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829421.142549:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626937600:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829421.142572:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626937728:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829421.142601:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829421.142630:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829421/real 1787829421]  req@ffff8cd0d48f6580 x1874671626937728/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829446 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829421.142643:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626937728/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829446 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829421.142651:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829421.142653:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829421.142656:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626937728:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829421.142666:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626937856:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829421.142682:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829421.142691:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829421/real 1787829421]  req@ffff8cd0d48f7480 x1874671626937856/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829446 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829421.142698:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626937856/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829446 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829421.142704:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829421.142706:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829421.142708:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626937856:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829421.334148:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829421.718218:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829422.358188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829422.742230:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829423.382250:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829423.766259:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829424.406178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829424.790240:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829425.430211:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829425.814314:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787829426.262202:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829426.262214:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787829426.262219:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829426.262231:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3312
00000100:00080000:0.0:1787829426.262243:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829426.262250:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829426.262300:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626937984/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829426.262329:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829426.262332:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829426.262334:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829426.262337:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3312
00000100:00080000:0.0:1787829426.262340:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829426.262360:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829426.262377:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829426.262382:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626937984:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829426.262393:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626938112/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829426.262404:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829426.262406:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787829426.262408:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829426.262411:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3312
00000100:00080000:0.0:1787829426.262416:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829426.262420:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829426.262426:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829426.262434:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626938240/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829426.262442:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829426.262446:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829426.262447:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829426.262449:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3312
00000100:00080000:0.0:1787829426.262455:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829426.262458:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829426.262462:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:1.0:1787829426.262467:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829426.262471:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626938368/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1787829426.262515:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829426/real 1787829426]  req@ffff8cd0c8e9de00 x1874671626937984/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829451 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829426.262530:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626937984/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829451 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829426.262539:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829426.262542:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829426.262545:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626937984:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829426.262567:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626938112:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829426.262599:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829426.262609:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829426/real 1787829426]  req@ffff8cd0c8e9e940 x1874671626938112/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829451 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829426.262630:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626938112/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829451 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829426.262636:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829426.262638:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829426.262641:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626938112:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787829426.262646:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626938240:192.168.203.145@tcp:8:kworker.0
00000800:00000100:1.0:1787829426.262662:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829426.262669:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829426/real 1787829426]  req@ffff8cd0c8e9c3c0 x1874671626938240/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829451 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829426.262676:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626938240/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829451 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829426.262681:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829426.262683:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829426.262685:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626938240:192.168.203.145@tcp:8:kworker.0
00000100:00100000:1.0:1787829426.262689:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626938368:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829426.262700:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829426.262706:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829426/real 1787829426]  req@ffff8cd0c8e9d680 x1874671626938368/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829451 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829426.262713:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626938368/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829451 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829426.262726:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829426.262729:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829426.262731:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626938368:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829426.454208:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829426.838308:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829427.478228:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829427.862239:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829428.502202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829428.886297:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829429.526183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829429.910269:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829430.550198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829430.934291:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787829431.382247:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829431.382284:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787829431.382290:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829431.382304:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3317
00000100:00080000:0.0:1787829431.382319:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829431.382329:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829431.382385:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d2c0 x1874671626938496/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829431.382418:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829431.382422:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829431.382425:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829431.382429:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3317
00000100:00080000:0.0:1787829431.382434:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829431.382439:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829431.382451:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829431.382465:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626938624/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829431.382477:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829431.382483:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:1.0:1787829431.382484:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626938496:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787829431.382486:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829431.382495:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3317
00000100:00080000:0.0:1787829431.382500:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829431.382505:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829431.382511:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829431.382525:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626938752/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829431.382562:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829431.382565:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829431.382568:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829431.382573:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3317
00000100:00080000:0.0:1787829431.382578:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829431.382583:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:1.0:1787829431.382589:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:0.0:1787829431.382592:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829431.382609:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626938880/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1787829431.382653:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829431/real 1787829431]  req@ffff8cd0c8e9d2c0 x1874671626938496/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829456 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829431.382676:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d2c0 x1874671626938496/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829456 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829431.382688:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829431.382704:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829431.382708:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626938496:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829431.382739:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626938624:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829431.382764:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829431.382778:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829431/real 1787829431]  req@ffff8cd0c8e9cb40 x1874671626938624/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829456 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829431.382791:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626938624/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829456 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829431.382800:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829431.382804:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829431.382808:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626938624:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787829431.382823:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626938752:192.168.203.145@tcp:8:kworker.0
00000800:00000100:1.0:1787829431.382846:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829431.382871:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829431/real 1787829431]  req@ffff8cd0c8e9f480 x1874671626938752/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829456 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829431.382884:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626938752/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829456 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829431.382892:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829431.382896:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829431.382899:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626938752:192.168.203.145@tcp:8:kworker.0
00000100:00100000:1.0:1787829431.382905:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626938880:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829431.382926:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829431.382938:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829431/real 1787829431]  req@ffff8cd0c8e9f840 x1874671626938880/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829456 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829431.382949:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626938880/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829456 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829431.382959:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829431.382962:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829431.382965:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626938880:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829431.574165:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829431.958261:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829432.598137:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829432.982332:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829433.622147:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829434.006236:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829434.646151:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829435.030306:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829435.670138:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829436.054235:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829436.502220:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829436.502228:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829436.502232:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829436.502264:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3322
00000100:00080000:1.0:1787829436.502274:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829436.502279:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829436.502310:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626939008/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829436.502334:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829436.502337:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829436.502338:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829436.502341:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3322
00000100:00080000:1.0:1787829436.502343:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829436.502345:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829436.502353:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829436.502362:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626939136/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829436.502370:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829436.502374:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829436.502376:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1787829436.502377:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626939008:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787829436.502380:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3322
00000100:00080000:1.0:1787829436.502386:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829436.502389:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829436.502392:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829436.502401:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626939264/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829436.502418:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829436.502420:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829436.502422:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829436.502424:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3322
00000100:00080000:1.0:1787829436.502429:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829436.502431:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829436.502435:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1787829436.502439:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829436.502448:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626939392/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829436.502478:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829436/real 1787829436]  req@ffff8cd0d48f7480 x1874671626939008/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829461 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829436.502492:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626939008/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829461 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829436.502512:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829436.502515:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829436.502518:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626939008:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829436.502536:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626939136:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829436.502554:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829436.502563:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829436/real 1787829436]  req@ffff8cd0d48f6580 x1874671626939136/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829461 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829436.502578:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626939136/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829461 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829436.502584:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829436.502586:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829436.502588:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626939136:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829436.502593:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626939264:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829436.502607:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829436.502613:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829436/real 1787829436]  req@ffff8cd0d48f4b40 x1874671626939264/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829461 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829436.502621:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626939264/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829461 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829436.502628:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829436.502631:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829436.502634:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626939264:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829436.502640:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626939392:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829436.502660:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829436.502672:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829436/real 1787829436]  req@ffff8cd0d48f4000 x1874671626939392/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829461 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829436.502684:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626939392/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829461 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829436.502706:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829436.502710:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829436.502714:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626939392:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829436.694139:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829437.078268:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829437.718177:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829438.102261:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829438.742166:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829439.126271:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829439.766194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829440.150291:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829440.790202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829441.174303:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829441.622247:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829441.622256:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829441.622260:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829441.622268:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3327
00000100:00080000:2.0:1787829441.622278:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829441.622285:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829441.622326:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626939520/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829441.622353:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829441.622357:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829441.622359:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829441.622363:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3327
00000100:00080000:2.0:1787829441.622367:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829441.622372:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829441.622390:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626939520:192.168.203.145@tcp:250:kworker.0
00000008:00100000:2.0:1787829441.622399:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829441.622418:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8b40 x1874671626939648/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829441.622432:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829441.622435:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829441.622437:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829441.622440:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3327
00000800:00000100:0.0:1787829441.622444:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787829441.622445:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829441.622452:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829441.622457:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829441.622483:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626939776/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829441.622491:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829441.622494:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829441.622495:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1787829441.622498:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829441/real 1787829441]  req@ffff8cd0c3da8f00 x1874671626939520/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829466 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829441.622501:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3327
00000100:00080000:2.0:1787829441.622508:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829441.622511:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829441.622515:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829441.622526:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626939904/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829441.622530:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626939520/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829466 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829441.622550:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829441.622554:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829441.622557:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626939520:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829441.622577:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626939648:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829441.622595:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829441.622604:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829441/real 1787829441]  req@ffff8cd0c3da8b40 x1874671626939648/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829466 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829441.622612:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8b40 x1874671626939648/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829466 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829441.622619:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829441.622621:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829441.622623:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626939648:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829441.622633:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626939776:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829441.622650:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829441.622662:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829441/real 1787829441]  req@ffff8cd0c3da83c0 x1874671626939776/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829466 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829441.622685:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626939776/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829466 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829441.622693:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829441.622696:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829441.622698:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626939776:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829441.622703:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626939904:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829441.622718:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829441.622727:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829441/real 1787829441]  req@ffff8cd0c3da9680 x1874671626939904/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829466 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829441.622734:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626939904/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829466 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829441.622740:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829441.622742:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829441.622744:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626939904:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829441.814173:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829442.198229:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829442.838196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829443.222257:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829443.862195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829444.246258:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829444.886202:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829445.270211:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829445.910170:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829446.294264:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829446.742234:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829446.742245:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829446.742249:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829446.742260:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3332
00000100:00080000:1.0:1787829446.742272:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829446.742279:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829446.742324:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626940032/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829446.742351:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829446.742354:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829446.742355:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829446.742358:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3332
00000100:00080000:1.0:1787829446.742360:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829446.742363:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829446.742393:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829446.742405:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626940160/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829446.742421:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829446.742427:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829446.742439:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829446.742449:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3332
00000100:00100000:0.0:1787829446.742453:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626940032:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787829446.742456:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829446.742468:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829446.742475:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829446.742492:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626940288/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829446.742503:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829446.742506:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829446.742508:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829446.742512:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3332
00000100:00080000:1.0:1787829446.742542:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829446.742546:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829446.742553:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829446.742565:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626940416/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829446.742592:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626940160:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829446.742640:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626940288:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829446.742658:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626940416:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829446.934250:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829447.318297:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829447.958177:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829448.342272:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829448.982213:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829449.366274:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:3.0:1787829449.814432:0:2343:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:3.0:1787829449.814488:0:2343:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:3.0:1787829449.814510:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:3.0:1787829449.814566:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:3.0:1787829449.814576:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:3.0:1787829449.814588:0:2343:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:0.0:1787829449.814649:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829446/real 1787829449]  req@ffff8cd0d48f5a40 x1874671626940160/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829471 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829449.814686:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626940160/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829471 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829449.814700:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829449.814705:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829449.814710:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626940160:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829449.814719:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829446/real 1787829449]  req@ffff8cd0d48f43c0 x1874671626940288/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829471 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829449.814735:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626940288/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829471 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829449.814743:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829449.814745:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829449.814748:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626940288:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829449.814753:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829446/real 1787829449]  req@ffff8cd0d48f70c0 x1874671626940416/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829471 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829449.814765:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626940416/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829471 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829449.814797:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829449.814801:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829449.814805:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626940416:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829449.814812:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829446/real 1787829449]  req@ffff8cd0d48f7840 x1874671626940032/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829471 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829449.814835:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626940032/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829471 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829449.814847:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829449.814851:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829449.814855:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626940032:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829450.006173:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829450.390243:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829451.030209:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829451.414200:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787829451.862213:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829451.862231:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787829451.862235:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829451.862246:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3337
00000100:00080000:0.0:1787829451.862258:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829451.862264:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829451.862294:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626940544/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829451.862320:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829451.862323:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829451.862325:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829451.862327:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3337
00000100:00080000:0.0:1787829451.862344:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829451.862348:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829451.862356:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829451.862371:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626940672/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1787829451.862381:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626940544:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787829451.862384:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829451.862391:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787829451.862413:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829451.862418:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3337
00000100:00080000:0.0:1787829451.862420:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829451.862424:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829451.862434:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829451.862448:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626940800/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829451.862472:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829451.862476:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829451.862477:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829451.862483:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3337
00000100:00080000:0.0:1787829451.862501:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829451.862508:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829451.862514:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829451.862528:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626940928/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1787829451.862571:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829451.862637:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829451/real 1787829451]  req@ffff8cd0c8e9c000 x1874671626940544/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829451.862661:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626940544/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829451.862675:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829451.862680:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829451.862686:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626940544:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829451.862719:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626940672:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829451.862745:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829451.862761:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829451/real 1787829451]  req@ffff8cd0c8e9f840 x1874671626940672/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829451.862776:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626940672/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829451.862785:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829451.862789:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829451.862793:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626940672:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787829451.862801:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626940800:192.168.203.145@tcp:8:kworker.0
00000800:00000100:1.0:1787829451.862826:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829451.862839:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829451/real 1787829451]  req@ffff8cd0c8e9f480 x1874671626940800/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829451.862870:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626940800/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829451.862879:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829451.862882:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829451.862885:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626940800:192.168.203.145@tcp:8:kworker.0
00000100:00100000:1.0:1787829451.862892:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626940928:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829451.862918:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829451.862931:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829451/real 1787829451]  req@ffff8cd0c8e9cb40 x1874671626940928/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829451.862945:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626940928/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829451.862953:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829451.862957:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829451.862960:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626940928:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829452.054190:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829452.438299:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829453.078154:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829453.462249:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829454.102179:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829454.486312:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829455.126184:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829455.510256:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829456.150154:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829456.534269:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787829456.982202:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829456.982212:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787829456.982217:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829456.982227:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3342
00000100:00080000:0.0:1787829456.982238:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829456.982247:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829456.982283:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d2c0 x1874671626941056/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829456.982313:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829456.982318:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829456.982320:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829456.982324:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3342
00000100:00080000:0.0:1787829456.982328:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829456.982332:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829456.982343:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829456.982370:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626941184/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829456.982383:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829456.982387:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787829456.982390:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829456.982394:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3342
00000100:00080000:0.0:1787829456.982398:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829456.982403:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829456.982422:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626941056:192.168.203.145@tcp:250:kworker.0
00000008:00100000:0.0:1787829456.982425:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829456.982451:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626941312/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829456.982466:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829456.982471:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829456.982473:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829456.982477:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3342
00000100:00080000:0.0:1787829456.982482:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829456.982488:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829456.982492:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829456.982505:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626941440/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1787829456.982534:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829456.982598:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829456/real 1787829456]  req@ffff8cd0c8e9d2c0 x1874671626941056/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829456.982628:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d2c0 x1874671626941056/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829456.982685:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829456.982690:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829456.982696:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626941056:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829456.982706:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626941184:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829456.982736:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829456.982753:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829456/real 1787829456]  req@ffff8cd0c8e9d680 x1874671626941184/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829456.982767:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626941184/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829456.982776:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829456.982780:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829456.982783:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626941184:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787829456.982818:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626941312:192.168.203.145@tcp:8:kworker.0
00000800:00000100:1.0:1787829456.982844:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829456.982857:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829456/real 1787829456]  req@ffff8cd0c8e9c3c0 x1874671626941312/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829456.982870:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626941312/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829456.982880:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829456.982884:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829456.982887:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626941312:192.168.203.145@tcp:8:kworker.0
00000100:00100000:1.0:1787829456.982903:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626941440:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829456.982930:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829456.982943:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829456/real 1787829456]  req@ffff8cd0c8e9e940 x1874671626941440/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829456.982956:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626941440/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829456.982965:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829456.982968:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829456.982972:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626941440:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829457.174190:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829457.558258:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829458.198178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829458.582276:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829459.222165:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829459.606226:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829460.246205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829460.630328:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829461.270178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829461.654216:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829462.102180:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829462.102192:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829462.102197:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829462.102207:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3347
00000100:00080000:2.0:1787829462.102218:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829462.102225:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829462.102263:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626941568/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829462.102291:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829462.102295:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829462.102297:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829462.102301:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3347
00000100:00080000:2.0:1787829462.102305:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829462.102309:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829462.102320:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829462.102344:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626941696/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1787829462.102350:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626941568:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787829462.102367:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829462.102383:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829462.102386:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829462.102392:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3347
00000100:00080000:2.0:1787829462.102397:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829462.102402:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829462.102409:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:1.0:1787829462.102412:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787829462.102430:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8b40 x1874671626941824/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829462.102443:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1787829462.102448:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829462/real 1787829462]  req@ffff8cd0c3da9680 x1874671626941568/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829487 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829462.102449:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829462.102454:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829462.102465:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3347
00000100:00080000:2.0:1787829462.102471:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1787829462.102473:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626941568/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829487 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829462.102480:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00080000:1.0:1787829462.102486:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:2.0:1787829462.102489:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00080000:1.0:1787829462.102492:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829462.102497:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626941568:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787829462.102509:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626941952/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1787829462.102528:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626941696:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829462.102548:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829462.102557:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829462/real 1787829462]  req@ffff8cd0c3da83c0 x1874671626941696/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829487 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829462.102572:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626941696/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829487 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829462.102578:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829462.102581:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829462.102583:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626941696:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787829462.102587:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626941824:192.168.203.145@tcp:8:kworker.0
00000800:00000100:1.0:1787829462.102602:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829462.102608:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829462/real 1787829462]  req@ffff8cd0c3da8b40 x1874671626941824/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829487 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829462.102616:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8b40 x1874671626941824/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829487 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829462.102621:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829462.102623:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829462.102625:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626941824:192.168.203.145@tcp:8:kworker.0
00000100:00100000:1.0:1787829462.102633:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626941952:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829462.102645:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829462.102655:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829462/real 1787829462]  req@ffff8cd0c3da8f00 x1874671626941952/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829487 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829462.102667:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626941952/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829487 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829462.102688:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829462.102691:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829462.102695:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626941952:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829462.294208:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829462.678231:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829463.318161:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829463.702193:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829464.342182:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829464.726266:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829465.366145:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829465.750213:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829466.390171:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829466.774261:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829467.222220:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829467.222230:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829467.222234:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829467.222244:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3353
00000100:00080000:1.0:1787829467.222255:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829467.222261:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829467.222313:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626942080/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829467.222344:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829467.222348:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829467.222350:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829467.222353:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3353
00000100:00080000:1.0:1787829467.222358:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829467.222363:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829467.222373:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829467.222387:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626942208/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829467.222399:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626942080:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787829467.222407:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829467.222412:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829467.222427:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829467.222434:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3353
00000100:00080000:1.0:1787829467.222439:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829467.222443:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829467.222449:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829467.222465:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626942336/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829467.222476:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829467.222480:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829467.222482:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829467.222487:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3353
00000100:00080000:1.0:1787829467.222491:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1787829467.222497:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787829467.222498:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829467.222508:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829467.222523:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626942464/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829467.222557:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829467/real 1787829467]  req@ffff8cd0d48f61c0 x1874671626942080/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829492 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829467.222579:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626942080/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829492 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829467.222590:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829467.222593:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829467.222596:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626942080:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829467.222623:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626942208:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829467.222661:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829467.222676:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829467/real 1787829467]  req@ffff8cd0d48f6d00 x1874671626942208/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829492 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829467.222690:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626942208/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829492 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829467.222700:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829467.222703:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829467.222705:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626942208:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829467.222710:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626942336:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829467.222729:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829467.222737:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829467/real 1787829467]  req@ffff8cd0d48f52c0 x1874671626942336/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829492 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829467.222745:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626942336/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829492 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829467.222751:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829467.222753:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829467.222755:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626942336:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829467.222759:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626942464:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829467.222770:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829467.222776:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829467/real 1787829467]  req@ffff8cd0d48f7c00 x1874671626942464/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829492 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829467.222795:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626942464/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829492 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829467.222801:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829467.222804:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829467.222806:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626942464:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829467.414167:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829467.798306:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829468.438199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829468.822313:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829469.462214:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829469.846297:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829470.486220:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829470.870226:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829471.510157:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829471.894305:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787829472.342229:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829472.342240:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787829472.342245:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829472.342266:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3358
00000100:00080000:3.0:1787829472.342278:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829472.342284:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787829472.342318:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626942592/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829472.342344:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829472.342346:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787829472.342348:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829472.342350:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3358
00000100:00080000:3.0:1787829472.342354:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829472.342357:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829472.342367:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829472.342384:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626942720/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829472.342405:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626942592:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787829472.342433:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829472.342438:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787829472.342441:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829472.342448:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3358
00000100:00080000:3.0:1787829472.342453:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829472.342458:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829472.342464:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829472.342479:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626942848/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829472.342486:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829472.342490:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787829472.342491:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829472.342494:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3358
00000100:00080000:3.0:1787829472.342498:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829472.342501:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829472.342504:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1787829472.342511:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829472.342518:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626942976/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829472.342584:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829472/real 1787829472]  req@ffff8cd0c8eacf00 x1874671626942592/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829497 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829472.342604:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626942592/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829497 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829472.342614:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829472.342618:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829472.342622:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626942592:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829472.342648:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626942720:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829472.342669:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829472.342679:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829472/real 1787829472]  req@ffff8cd0c8eac000 x1874671626942720/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829497 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829472.342688:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626942720/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829497 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829472.342728:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829472.342733:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829472.342737:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626942720:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829472.342744:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626942848:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829472.342771:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829472.342782:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829472/real 1787829472]  req@ffff8cd0c8eaf840 x1874671626942848/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829497 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829472.342791:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626942848/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829497 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829472.342797:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829472.342799:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829472.342801:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626942848:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829472.342805:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626942976:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829472.342820:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829472.342835:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829472/real 1787829472]  req@ffff8cd0c8eae940 x1874671626942976/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829497 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829472.342843:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626942976/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829497 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829472.342849:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829472.342851:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829472.342859:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626942976:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829472.534185:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829472.918217:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829473.558217:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829473.942356:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829474.582214:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829474.966215:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829475.606175:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829475.990287:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829476.630264:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829477.014267:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787829477.462237:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829477.462247:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787829477.462252:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829477.462264:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3363
00000100:00080000:3.0:1787829477.462278:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829477.462286:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787829477.462336:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626943104/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829477.462362:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829477.462364:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787829477.462366:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829477.462368:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3363
00000100:00080000:3.0:1787829477.462371:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829477.462374:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829477.462384:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829477.462397:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626943232/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829477.462408:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829477.462412:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787829477.462414:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829477.462418:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3363
00000100:00080000:3.0:1787829477.462423:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829477.462427:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829477.462442:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626943104:192.168.203.145@tcp:250:kworker.0
00000008:00100000:3.0:1787829477.462458:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829477.462478:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626943360/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829477.462492:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829477.462496:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787829477.462499:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829477.462505:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3363
00000100:00080000:3.0:1787829477.462510:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829477.462515:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829477.462521:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1787829477.462530:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:3.0:1787829477.462535:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626943488/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829477.462594:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829477/real 1787829477]  req@ffff8cd0c8ead2c0 x1874671626943104/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829502 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829477.462619:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626943104/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829502 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829477.462630:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829477.462634:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829477.462637:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626943104:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829477.462644:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626943232:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829477.462664:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829477.462674:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829477/real 1787829477]  req@ffff8cd0c8eae1c0 x1874671626943232/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829502 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829477.462687:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626943232/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829502 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829477.462695:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829477.462700:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829477.462704:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626943232:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829477.462731:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626943360:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829477.462750:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829477.462777:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829477/real 1787829477]  req@ffff8cd0c8eada40 x1874671626943360/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829502 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829477.462791:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626943360/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829502 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829477.462800:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829477.462804:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829477.462808:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626943360:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829477.462814:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626943488:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829477.462835:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829477.462847:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829477/real 1787829477]  req@ffff8cd0c8eac780 x1874671626943488/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829502 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829477.462861:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626943488/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829502 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829477.462870:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829477.462874:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829477.462877:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626943488:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829477.654151:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829478.038310:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829478.678196:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829479.062245:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829479.702228:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829480.086229:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829480.726169:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829481.110232:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829481.750199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829482.134265:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829482.582171:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829482.582191:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829482.582195:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829482.582205:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3368
00000100:00080000:2.0:1787829482.582213:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829482.582220:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829482.582257:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626943616/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829482.582281:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829482.582284:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829482.582285:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829482.582288:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3368
00000100:00080000:2.0:1787829482.582291:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829482.582294:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829482.582302:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829482.582312:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626943744/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829482.582318:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626943616:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787829482.582325:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829482.582328:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829482.582329:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829482.582332:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3368
00000100:00080000:2.0:1787829482.582334:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829482.582337:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829482.582351:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829482.582362:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626943872/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829482.582372:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829482.582375:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829482.582376:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829482.582382:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3368
00000800:00000100:0.0:1787829482.582387:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787829482.582389:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829482.582397:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829482.582404:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829482.582419:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626944000/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829482.582427:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829482/real 1787829482]  req@ffff8cd0c3daa940 x1874671626943616/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829507 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829482.582449:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626943616/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829507 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829482.582463:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829482.582467:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829482.582471:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626943616:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829482.582491:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626943744:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829482.582509:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829482.582518:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829482/real 1787829482]  req@ffff8cd0c3da8780 x1874671626943744/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829507 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829482.582526:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626943744/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829507 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829482.582532:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829482.582534:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829482.582536:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626943744:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829482.582540:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626943872:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829482.582551:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829482.582557:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829482/real 1787829482]  req@ffff8cd0c3da9a40 x1874671626943872/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829507 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829482.582564:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626943872/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829507 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829482.582569:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829482.582571:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829482.582573:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626943872:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829482.582581:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626944000:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829482.582594:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829482.582601:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829482/real 1787829482]  req@ffff8cd0c3dab0c0 x1874671626944000/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829507 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829482.582609:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626944000/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829507 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829482.582614:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829482.582616:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829482.582618:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626944000:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829482.774273:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829483.158250:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829483.798230:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829484.182250:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829484.822161:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829485.206212:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829485.846201:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829486.230230:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829486.870167:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829487.254228:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829487.702186:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829487.702194:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829487.702197:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829487.702205:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3373
00000100:00080000:1.0:1787829487.702215:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829487.702220:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829487.702252:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626944128/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829487.702290:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829487.702293:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829487.702294:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829487.702298:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3373
00000100:00080000:1.0:1787829487.702301:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829487.702303:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829487.702310:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829487.702320:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626944256/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829487.702328:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829487.702331:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829487.702332:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829487.702335:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3373
00000100:00080000:1.0:1787829487.702337:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829487.702344:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829487.702349:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626944128:192.168.203.145@tcp:250:kworker.0
00000008:00100000:1.0:1787829487.702350:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829487.702361:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626944384/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829487.702369:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829487.702371:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829487.702372:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829487.702375:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3373
00000100:00080000:1.0:1787829487.702385:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829487.702388:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829487.702392:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829487.702401:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626944512/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787829487.702415:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829487.702453:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829487/real 1787829487]  req@ffff8cd0d48f7c00 x1874671626944128/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829512 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829487.702471:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626944128/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829512 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829487.702483:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829487.702487:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829487.702491:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626944128:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829487.702499:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626944256:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829487.702525:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829487.702539:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829487/real 1787829487]  req@ffff8cd0d48f52c0 x1874671626944256/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829512 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829487.702563:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626944256/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829512 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829487.702582:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829487.702586:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829487.702590:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626944256:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829487.702616:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626944384:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829487.702639:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829487.702652:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829487/real 1787829487]  req@ffff8cd0d48f6d00 x1874671626944384/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829512 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829487.702664:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626944384/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829512 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829487.702672:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829487.702676:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829487.702679:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626944384:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829487.702686:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626944512:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829487.702706:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829487.702718:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829487/real 1787829487]  req@ffff8cd0d48f61c0 x1874671626944512/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829512 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829487.702727:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626944512/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829512 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829487.702741:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829487.702745:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829487.702748:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626944512:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829487.894170:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829488.278251:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829488.918229:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829489.302271:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829489.942193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829490.326263:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829490.966214:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829491.350325:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829491.990204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829492.374208:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829492.822167:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829492.822176:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829492.822179:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829492.822187:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3378
00000100:00080000:1.0:1787829492.822196:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829492.822201:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829492.822233:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626944640/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829492.822254:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829492.822257:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829492.822259:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829492.822263:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3378
00000100:00080000:1.0:1787829492.822268:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829492.822273:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829492.822283:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829492.822295:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626944640:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829492.822299:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626944768/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829492.822314:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829492.822332:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829492.822335:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829492.822340:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3378
00000100:00080000:1.0:1787829492.822345:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000800:00000100:0.0:1787829492.822351:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787829492.822352:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829492.822361:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829492.822376:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626944896/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829492.822388:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1787829492.822389:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829492/real 1787829492]  req@ffff8cd0d48f7840 x1874671626944640/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829492.822397:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829492.822402:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829492.822410:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3378
00000100:00100000:0.0:1787829492.822411:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626944640/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829492.822416:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829492.822423:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829492.822426:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00080000:0.0:1787829492.822429:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829492.822435:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626944640:192.168.203.145@tcp:250:kworker.0
00000008:00100000:1.0:1787829492.822437:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829492.822453:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626945024/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829492.822456:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626944768:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829492.822480:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829492.822491:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829492/real 1787829492]  req@ffff8cd0d48f70c0 x1874671626944768/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829492.822500:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626944768/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829492.822506:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829492.822508:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829492.822510:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626944768:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829492.822521:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626944896:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829492.822535:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829492.822542:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829492/real 1787829492]  req@ffff8cd0d48f43c0 x1874671626944896/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829492.822549:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626944896/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829492.822554:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829492.822556:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829492.822558:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626944896:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829492.822568:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626945024:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829492.822578:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829492.822584:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829492/real 1787829492]  req@ffff8cd0d48f5a40 x1874671626945024/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829492.822591:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626945024/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829492.822596:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829492.822598:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829492.822600:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626945024:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829493.014209:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829493.398218:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829494.038195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829494.422300:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829495.062200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829495.446298:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829496.086207:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829496.470304:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829497.110223:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829497.494213:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829497.942199:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829497.942209:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829497.942213:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829497.942223:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3383
00000100:00080000:1.0:1787829497.942234:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829497.942241:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829497.942276:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626945152/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829497.942318:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829497.942321:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829497.942323:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829497.942328:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3383
00000100:00080000:1.0:1787829497.942332:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829497.942337:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829497.942347:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829497.942361:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626945280/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829497.942372:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829497.942376:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829497.942378:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829497.942382:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3383
00000100:00080000:1.0:1787829497.942385:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829497.942389:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829497.942395:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829497.942411:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626945152:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829497.942411:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626945408/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829497.942435:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829497.942439:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829497.942442:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829497.942447:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3383
00000100:00080000:1.0:1787829497.942460:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829497.942466:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829497.942472:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829497.942487:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626945536/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787829497.942510:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829497.942564:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829497/real 1787829497]  req@ffff8cd0d48f4000 x1874671626945152/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829522 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829497.942595:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626945152/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829522 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829497.942604:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829497.942608:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829497.942610:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626945152:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829497.942617:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626945280:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829497.942634:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829497.942642:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829497/real 1787829497]  req@ffff8cd0d48f4b40 x1874671626945280/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829522 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829497.942653:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626945280/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829522 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829497.942661:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829497.942664:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829497.942668:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626945280:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829497.942696:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626945408:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829497.942715:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829497.942725:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829497/real 1787829497]  req@ffff8cd0d48f6580 x1874671626945408/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829522 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829497.942737:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626945408/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829522 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829497.942755:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829497.942759:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829497.942762:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626945408:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829497.942768:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626945536:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829497.942784:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829497.942793:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829497/real 1787829497]  req@ffff8cd0d48f7480 x1874671626945536/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829522 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829497.942803:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626945536/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829522 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829497.942811:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829497.942813:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829497.942816:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626945536:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829498.134139:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829498.518278:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829499.158158:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829499.542228:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829500.182159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829500.566249:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829501.206129:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829501.590228:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829502.230139:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829502.614304:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787829503.062158:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829503.062167:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787829503.062170:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829503.062178:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3388
00000100:00080000:3.0:1787829503.062188:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829503.062194:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787829503.062225:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626945664/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829503.062267:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829503.062271:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787829503.062273:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829503.062277:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3388
00000100:00080000:3.0:1787829503.062282:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829503.062287:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829503.062297:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829503.062311:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626945664:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829503.062314:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626945792/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829503.062326:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829503.062329:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787829503.062331:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829503.062333:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3388
00000100:00080000:3.0:1787829503.062337:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829503.062340:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829503.062344:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829503.062354:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626945920/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829503.062361:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829503.062364:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787829503.062365:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829503.062368:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3388
00000800:00000100:0.0:1787829503.062372:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787829503.062380:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829503.062385:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829503.062389:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829503.062398:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626946048/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829503.062428:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829503/real 1787829503]  req@ffff8cd0c8eac3c0 x1874671626945664/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829528 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829503.062445:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626945664/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829528 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829503.062454:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829503.062457:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829503.062460:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626945664:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829503.062484:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626945792:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829503.062506:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829503.062521:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829503/real 1787829503]  req@ffff8cd0c8eaf0c0 x1874671626945792/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829528 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829503.062535:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626945792/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829528 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829503.062545:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829503.062549:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829503.062553:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626945792:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829503.062559:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626945920:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829503.062582:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829503.062595:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829503/real 1787829503]  req@ffff8cd0c8eac780 x1874671626945920/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829528 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829503.062607:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626945920/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829528 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829503.062613:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829503.062616:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829503.062619:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626945920:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829503.062627:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626946048:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829503.062648:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829503.062661:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829503/real 1787829503]  req@ffff8cd0c8eada40 x1874671626946048/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829528 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829503.062683:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626946048/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829528 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829503.062689:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829503.062691:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829503.062693:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626946048:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829503.254167:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829503.638234:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829504.278162:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829504.662229:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829505.302206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829505.686221:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829506.326160:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829506.710216:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829507.350181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829507.734230:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787829508.182158:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829508.182167:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787829508.182170:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829508.182179:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3394
00000100:00080000:3.0:1787829508.182189:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829508.182194:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787829508.182234:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626946176/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829508.182257:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829508.182259:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787829508.182261:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829508.182263:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3394
00000100:00080000:3.0:1787829508.182265:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829508.182268:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829508.182289:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829508.182298:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626946304/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829508.182307:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829508.182309:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787829508.182311:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829508.182314:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3394
00000100:00080000:3.0:1787829508.182317:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1787829508.182320:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626946176:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787829508.182321:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829508.182327:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829508.182335:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626946432/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829508.182342:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829508.182345:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787829508.182346:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829508.182349:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3394
00000100:00080000:3.0:1787829508.182351:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829508.182353:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829508.182356:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829508.182363:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626946560/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787829508.182389:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829508.182437:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829508/real 1787829508]  req@ffff8cd0c8eae1c0 x1874671626946176/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829533 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829508.182457:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626946176/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829533 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829508.182470:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829508.182474:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829508.182479:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626946176:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829508.182487:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626946304:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829508.182510:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829508.182524:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829508/real 1787829508]  req@ffff8cd0c8ead2c0 x1874671626946304/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829533 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829508.182538:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626946304/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829533 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829508.182562:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829508.182566:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829508.182570:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626946304:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829508.182593:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626946432:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829508.182622:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829508.182635:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829508/real 1787829508]  req@ffff8cd0c8eae940 x1874671626946432/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829533 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829508.182648:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626946432/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829533 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829508.182657:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829508.182661:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829508.182665:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626946432:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829508.182672:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626946560:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829508.182690:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829508.182701:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829508/real 1787829508]  req@ffff8cd0c8eaf840 x1874671626946560/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829533 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829508.182714:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626946560/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829533 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829508.182723:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829508.182726:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829508.182737:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626946560:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829508.374163:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829508.758315:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829509.398178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829509.782225:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829510.422168:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829510.806244:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829511.446223:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829511.830277:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829512.470175:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829512.854248:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787829513.302180:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829513.302188:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787829513.302192:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829513.302199:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3399
00000100:00080000:3.0:1787829513.302208:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829513.302213:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787829513.302256:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626946688/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829513.302284:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829513.302286:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787829513.302288:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829513.302291:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3399
00000100:00080000:3.0:1787829513.302293:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829513.302296:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829513.302304:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829513.302314:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626946816/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829513.302325:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626946688:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787829513.302329:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829513.302333:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787829513.302347:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829513.302351:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3399
00000100:00080000:3.0:1787829513.302355:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829513.302358:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829513.302363:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829513.302373:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626946944/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829513.302380:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829513.302382:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787829513.302384:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829513.302386:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3399
00000100:00080000:3.0:1787829513.302389:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829513.302392:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829513.302396:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829513.302404:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626947072/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829513.302424:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626946816:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829513.302451:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626946944:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829513.302468:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626947072:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829513.494200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829513.878241:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829514.518171:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829514.902248:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829515.542205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829515.926222:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:1.0:1787829516.374375:0:2345:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:1.0:1787829516.374419:0:2345:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1787829516.374436:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787829516.374482:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787829516.374491:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787829516.374500:0:2345:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:0.0:1787829516.374581:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829513/real 1787829516]  req@ffff8cd0c8eacf00 x1874671626946816/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829538 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829516.374605:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626946816/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829538 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829516.374619:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829516.374625:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829516.374629:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626946816:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829516.374637:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829513/real 1787829516]  req@ffff8cd0c8eaf840 x1874671626946944/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829538 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829516.374650:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626946944/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829538 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829516.374659:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829516.374662:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829516.374666:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626946944:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829516.374673:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829513/real 1787829516]  req@ffff8cd0c8eae940 x1874671626947072/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829538 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829516.374697:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626947072/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829538 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829516.374706:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829516.374711:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829516.374714:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626947072:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829516.374722:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829513/real 1787829516]  req@ffff8cd0c8eac000 x1874671626946688/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829538 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829516.374736:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626946688/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829538 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829516.374745:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829516.374749:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829516.374753:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626946688:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829516.566186:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829516.950291:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829517.590184:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829517.974262:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829518.422219:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829518.422235:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829518.422239:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829518.422247:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3404
00000100:00080000:2.0:1787829518.422256:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829518.422263:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829518.422295:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626947200/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829518.422316:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829518.422318:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829518.422320:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829518.422322:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3404
00000100:00080000:2.0:1787829518.422325:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829518.422327:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829518.422334:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829518.422343:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626947328/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829518.422350:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829518.422353:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829518.422354:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829518.422356:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3404
00000100:00080000:2.0:1787829518.422359:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1787829518.422373:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626947200:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787829518.422383:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829518.422390:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829518.422406:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626947456/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829518.422419:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829518.422423:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829518.422424:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829518.422429:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3404
00000100:00080000:2.0:1787829518.422434:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829518.422438:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829518.422443:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829518.422455:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626947584/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787829518.422460:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829518.422510:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829518/real 1787829518]  req@ffff8cd0c3dabc00 x1874671626947200/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829543 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829518.422530:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626947200/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829543 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829518.422542:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829518.422546:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829518.422551:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626947200:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829518.422566:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626947328:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829518.422595:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829518.422610:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829518/real 1787829518]  req@ffff8cd0c3daa580 x1874671626947328/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829543 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829518.422624:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626947328/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829543 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829518.422633:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829518.422637:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829518.422641:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626947328:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829518.422665:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626947456:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829518.422687:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829518.422700:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829518/real 1787829518]  req@ffff8cd0c3dab840 x1874671626947456/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829543 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829518.422714:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626947456/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829543 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829518.422723:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829518.422727:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829518.422731:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626947456:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829518.422738:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626947584:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829518.422758:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829518.422771:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829518/real 1787829518]  req@ffff8cd0c3da8000 x1874671626947584/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829543 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829518.422796:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626947584/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829543 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829518.422806:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829518.422810:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829518.422814:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626947584:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829518.614191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829518.998276:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829519.638203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829520.022260:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829520.662250:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829521.046256:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829521.686165:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829522.070200:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829522.710218:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829523.094232:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787829523.542214:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829523.542222:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787829523.542225:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829523.542233:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3409
00000100:00080000:0.0:1787829523.542243:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829523.542248:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829523.542280:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626947712/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829523.542306:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829523.542309:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829523.542310:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829523.542313:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3409
00000100:00080000:0.0:1787829523.542315:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829523.542317:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829523.542325:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829523.542334:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626947840/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1787829523.542355:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626947712:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787829523.542365:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829523.542368:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787829523.542369:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829523.542373:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3409
00000100:00080000:0.0:1787829523.542376:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829523.542379:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829523.542383:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829523.542393:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626947968/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829523.542400:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829523.542403:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829523.542404:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829523.542406:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3409
00000100:00080000:0.0:1787829523.542408:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829523.542411:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:1.0:1787829523.542415:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:0.0:1787829523.542418:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829523.542428:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626948096/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1787829523.542575:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829523/real 1787829523]  req@ffff8cd0c8e9de00 x1874671626947712/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829548 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829523.542594:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626947712/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829548 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829523.542603:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829523.542607:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829523.542610:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626947712:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829523.542646:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626947840:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829523.542691:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829523.542701:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829523/real 1787829523]  req@ffff8cd0c8e9cf00 x1874671626947840/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829548 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829523.542720:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626947840/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829548 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829523.542725:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829523.542728:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829523.542730:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626947840:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787829523.542735:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626947968:192.168.203.145@tcp:8:kworker.0
00000800:00000100:1.0:1787829523.542750:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829523.542757:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829523/real 1787829523]  req@ffff8cd0c8e9fc00 x1874671626947968/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829548 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829523.542764:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626947968/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829548 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829523.542769:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829523.542770:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829523.542772:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626947968:192.168.203.145@tcp:8:kworker.0
00000100:00100000:1.0:1787829523.542776:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626948096:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829523.542787:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829523.542792:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829523/real 1787829523]  req@ffff8cd0c8e9ed00 x1874671626948096/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829548 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829523.542798:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626948096/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829548 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829523.542811:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829523.542815:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829523.542818:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626948096:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829523.734151:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829524.118285:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829524.758206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829525.142232:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829525.782175:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829526.166256:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829526.806209:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829527.190305:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829527.830192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829528.214242:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787829528.662173:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829528.662183:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787829528.662187:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829528.662198:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3414
00000100:00080000:0.0:1787829528.662226:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829528.662231:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829528.662262:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626948224/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829528.662289:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829528.662291:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829528.662293:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829528.662295:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3414
00000100:00080000:0.0:1787829528.662298:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829528.662300:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829528.662307:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829528.662316:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626948352/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829528.662323:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829528.662325:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787829528.662327:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829528.662328:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3414
00000100:00080000:0.0:1787829528.662330:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829528.662332:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829528.662335:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829528.662340:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626948480/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829528.662347:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829528.662354:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829528.662355:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829528.662359:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3414
00000100:00080000:0.0:1787829528.662362:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829528.662365:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829528.662368:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829528.662373:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626948224:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829528.662380:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626948608/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1787829528.662432:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829528.662464:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829528/real 1787829528]  req@ffff8cd0c8e9e580 x1874671626948224/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829553 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829528.662477:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626948224/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829553 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829528.662485:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829528.662488:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829528.662491:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626948224:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829528.662496:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626948352:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829528.662509:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829528.662516:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829528/real 1787829528]  req@ffff8cd0c8e9ed00 x1874671626948352/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829553 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829528.662523:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626948352/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829553 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829528.662529:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829528.662530:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829528.662532:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626948352:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787829528.662543:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626948480:192.168.203.145@tcp:8:kworker.0
00000800:00000100:1.0:1787829528.662557:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829528.662564:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829528/real 1787829528]  req@ffff8cd0c8e9fc00 x1874671626948480/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829553 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829528.662571:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626948480/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829553 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829528.662577:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829528.662578:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829528.662580:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626948480:192.168.203.145@tcp:8:kworker.0
00000100:00100000:1.0:1787829528.662598:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626948608:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829528.662609:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829528.662615:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829528/real 1787829528]  req@ffff8cd0c8e9cf00 x1874671626948608/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829553 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829528.662622:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626948608/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829553 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829528.662627:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829528.662629:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829528.662631:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626948608:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829528.854209:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829529.238233:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829529.878185:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:2.0:1787829530.262289:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829530.902193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829531.286217:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829531.926156:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829532.310247:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829532.950198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829533.334278:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787829533.782199:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829533.782207:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787829533.782210:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829533.782219:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3419
00000100:00080000:0.0:1787829533.782229:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829533.782234:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829533.782265:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626948736/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829533.782287:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829533.782290:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829533.782291:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829533.782294:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3419
00000100:00080000:0.0:1787829533.782296:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829533.782298:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829533.782306:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829533.782329:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626948864/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829533.782338:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829533.782341:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:1.0:1787829533.782341:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626948736:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787829533.782344:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829533.782353:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3419
00000100:00080000:0.0:1787829533.782356:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829533.782360:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829533.782365:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829533.782383:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626948992/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829533.782389:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829533.782392:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829533.782393:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829533.782396:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3419
00000100:00080000:0.0:1787829533.782398:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829533.782401:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829533.782404:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:1.0:1787829533.782410:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829533.782414:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626949120/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1787829533.782463:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829533/real 1787829533]  req@ffff8cd0c8e9de00 x1874671626948736/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829558 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829533.782495:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626948736/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829558 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829533.782510:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829533.782515:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829533.782519:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626948736:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829533.782544:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626948864:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829533.782567:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829533.782580:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829533/real 1787829533]  req@ffff8cd0c8e9e940 x1874671626948864/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829558 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829533.782595:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626948864/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829558 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829533.782604:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829533.782608:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829533.782612:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626948864:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787829533.782618:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626948992:192.168.203.145@tcp:8:kworker.0
00000800:00000100:1.0:1787829533.782639:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829533.782650:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829533/real 1787829533]  req@ffff8cd0c8e9c3c0 x1874671626948992/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829558 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829533.782663:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626948992/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829558 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829533.782685:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829533.782689:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829533.782693:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626948992:192.168.203.145@tcp:8:kworker.0
00000100:00100000:1.0:1787829533.782699:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626949120:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829533.782720:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829533.782731:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829533/real 1787829533]  req@ffff8cd0c8e9d680 x1874671626949120/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829558 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829533.782744:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626949120/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829558 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829533.782753:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829533.782757:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829533.782760:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626949120:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829533.974224:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829534.358222:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829534.998226:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829535.382263:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829536.022171:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829536.406242:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829537.046172:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829537.430253:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829538.070181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829538.454221:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787829538.902220:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829538.902246:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787829538.902251:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829538.902262:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3424
00000100:00080000:0.0:1787829538.902273:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829538.902280:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829538.902313:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d2c0 x1874671626949248/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829538.902342:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829538.902346:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829538.902349:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829538.902353:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3424
00000100:00080000:0.0:1787829538.902357:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829538.902361:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829538.902372:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829538.902385:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626949376/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829538.902397:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829538.902401:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787829538.902403:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829538.902409:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3424
00000100:00080000:0.0:1787829538.902413:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829538.902417:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829538.902424:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829538.902429:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626949248:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829538.902440:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626949504/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829538.902465:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829538.902469:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829538.902472:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829538.902477:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3424
00000100:00080000:0.0:1787829538.902500:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829538.902505:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829538.902511:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:1.0:1787829538.902517:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829538.902534:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626949632/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1787829538.902575:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829538/real 1787829538]  req@ffff8cd0c8e9d2c0 x1874671626949248/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829563 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829538.902594:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d2c0 x1874671626949248/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829563 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829538.902603:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829538.902609:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829538.902612:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626949248:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829538.902629:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626949376:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829538.902648:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829538.902656:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829538/real 1787829538]  req@ffff8cd0c8e9cb40 x1874671626949376/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829563 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829538.902664:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626949376/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829563 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829538.902669:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829538.902671:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829538.902674:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626949376:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787829538.902699:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626949504:192.168.203.145@tcp:8:kworker.0
00000800:00000100:1.0:1787829538.902710:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829538.902717:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829538/real 1787829538]  req@ffff8cd0c8e9f480 x1874671626949504/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829563 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829538.902723:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626949504/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829563 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829538.902728:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829538.902730:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829538.902731:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626949504:192.168.203.145@tcp:8:kworker.0
00000100:00100000:1.0:1787829538.902735:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626949632:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829538.902745:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829538.902751:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829538/real 1787829538]  req@ffff8cd0c8e9f840 x1874671626949632/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829563 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829538.902763:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626949632/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829563 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829538.902768:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829538.902769:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829538.902771:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626949632:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829539.094200:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829539.478245:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829540.118247:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829540.502247:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829541.142161:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829541.526239:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829542.166192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829542.550251:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829543.190148:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829543.574247:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829544.022235:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829544.022243:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829544.022246:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829544.022254:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3429
00000100:00080000:2.0:1787829544.022264:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829544.022269:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829544.022316:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626949760/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829544.022340:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829544.022344:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829544.022346:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829544.022350:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3429
00000100:00080000:2.0:1787829544.022354:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829544.022359:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829544.022369:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829544.022381:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626949888/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1787829544.022386:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626949760:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787829544.022392:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829544.022396:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829544.022398:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829544.022401:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3429
00000100:00080000:2.0:1787829544.022404:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829544.022407:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829544.022412:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829544.022420:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626950016/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829544.022428:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829544.022431:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829544.022433:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829544.022442:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3429
00000800:00000100:1.0:1787829544.022445:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787829544.022446:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829544.022450:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829544.022455:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829544.022465:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daad00 x1874671626950144/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1787829544.022483:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829544/real 1787829544]  req@ffff8cd0c3da9e00 x1874671626949760/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829569 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829544.022500:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626949760/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829569 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829544.022531:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829544.022536:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829544.022540:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626949760:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829544.022563:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626949888:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829544.022588:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829544.022602:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829544/real 1787829544]  req@ffff8cd0c3da92c0 x1874671626949888/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829569 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829544.022615:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626949888/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829569 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829544.022639:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829544.022643:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829544.022647:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626949888:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787829544.022654:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626950016:192.168.203.145@tcp:8:kworker.0
00000800:00000100:1.0:1787829544.022678:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829544.022690:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829544/real 1787829544]  req@ffff8cd0c3daa1c0 x1874671626950016/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829569 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829544.022698:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626950016/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829569 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829544.022704:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829544.022707:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829544.022709:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626950016:192.168.203.145@tcp:8:kworker.0
00000100:00100000:1.0:1787829544.022713:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626950144:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829544.022727:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829544.022733:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829544/real 1787829544]  req@ffff8cd0c3daad00 x1874671626950144/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829569 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829544.022741:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daad00 x1874671626950144/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829569 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829544.022745:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829544.022747:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829544.022767:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626950144:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829544.214139:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829544.598188:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829545.238163:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829545.622272:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829546.262224:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829546.646204:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829547.286211:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829547.670336:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829548.310238:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829548.694175:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829549.142173:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829549.142184:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829549.142188:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829549.142198:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3434
00000100:00080000:2.0:1787829549.142209:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829549.142217:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829549.142268:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daad00 x1874671626950272/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829549.142291:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829549.142294:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829549.142295:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829549.142298:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3434
00000100:00080000:2.0:1787829549.142302:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829549.142307:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829549.142315:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829549.142324:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626950400/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829549.142332:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829549.142346:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829549.142348:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1787829549.142353:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626950272:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787829549.142353:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3434
00000100:00080000:2.0:1787829549.142359:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829549.142364:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829549.142369:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829549.142378:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626950528/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829549.142386:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829549.142389:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829549.142391:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829549.142393:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3434
00000100:00080000:2.0:1787829549.142396:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829549.142399:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829549.142402:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829549.142408:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626950656/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1787829549.142423:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829549.142464:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829549/real 1787829549]  req@ffff8cd0c3daad00 x1874671626950272/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829574 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829549.142484:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daad00 x1874671626950272/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829574 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829549.142495:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829549.142500:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829549.142504:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626950272:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829549.142513:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626950400:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829549.142534:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829549.142546:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829549/real 1787829549]  req@ffff8cd0c3daa1c0 x1874671626950400/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829574 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829549.142559:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626950400/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829574 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829549.142567:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829549.142571:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829549.142574:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626950400:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787829549.142595:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626950528:192.168.203.145@tcp:8:kworker.0
00000800:00000100:1.0:1787829549.142614:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829549.142636:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829549/real 1787829549]  req@ffff8cd0c3da92c0 x1874671626950528/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829574 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829549.142649:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626950528/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829574 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829549.142657:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829549.142660:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829549.142664:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626950528:192.168.203.145@tcp:8:kworker.0
00000100:00100000:1.0:1787829549.142670:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626950656:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829549.142691:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829549.142703:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829549/real 1787829549]  req@ffff8cd0c3da9e00 x1874671626950656/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829574 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829549.142716:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626950656/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829574 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829549.142738:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829549.142742:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829549.142746:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626950656:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829549.334157:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829549.718288:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829550.358184:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829550.742190:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829551.382187:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829551.766265:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829552.406186:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829552.790177:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829553.430181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829553.814280:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829554.262179:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829554.262189:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829554.262194:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829554.262204:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3440
00000100:00080000:2.0:1787829554.262215:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829554.262223:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829554.262261:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626950784/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829554.262303:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829554.262307:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829554.262309:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829554.262314:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3440
00000100:00080000:2.0:1787829554.262319:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829554.262325:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829554.262335:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829554.262343:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626950784:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787829554.262351:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626950912/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829554.262364:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829554.262368:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829554.262370:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829554.262374:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3440
00000100:00080000:2.0:1787829554.262378:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829554.262383:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829554.262388:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829554.262400:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626951040/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1787829554.262403:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787829554.262416:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829554.262420:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829554.262422:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829554.262427:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3440
00000100:00100000:1.0:1787829554.262438:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829554/real 1787829554]  req@ffff8cd0c3da8000 x1874671626950784/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829579 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829554.262439:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829554.262448:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829554.262457:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829554.262458:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626950784/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829579 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:2.0:1787829554.262474:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626951168/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829554.262475:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829554.262482:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829554.262490:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626950784:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829554.262513:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626950912:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829554.262531:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829554.262540:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829554/real 1787829554]  req@ffff8cd0c3dab840 x1874671626950912/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829579 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829554.262549:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626950912/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829579 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829554.262554:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829554.262557:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829554.262559:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626950912:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787829554.262562:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626951040:192.168.203.145@tcp:8:kworker.0
00000800:00000100:1.0:1787829554.262574:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829554.262580:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829554/real 1787829554]  req@ffff8cd0c3daa580 x1874671626951040/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829579 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829554.262587:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626951040/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829579 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829554.262592:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829554.262594:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829554.262596:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626951040:192.168.203.145@tcp:8:kworker.0
00000100:00100000:1.0:1787829554.262604:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626951168:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829554.262618:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829554.262624:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829554/real 1787829554]  req@ffff8cd0c3dabc00 x1874671626951168/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829579 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829554.262631:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626951168/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829579 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829554.262636:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829554.262638:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829554.262640:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626951168:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829554.454178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829554.838215:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829555.478162:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829555.862273:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829556.502169:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829556.886226:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829557.526240:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829557.910224:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829558.550143:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829558.934266:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829559.382238:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829559.382246:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829559.382250:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829559.382258:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3445
00000100:00080000:1.0:1787829559.382267:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829559.382273:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829559.382307:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7480 x1874671626951296/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829559.382334:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829559.382337:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829559.382338:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829559.382340:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3445
00000100:00080000:1.0:1787829559.382343:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829559.382359:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829559.382369:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829559.382382:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626951296:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829559.382383:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6580 x1874671626951424/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829559.382397:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829559.382400:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829559.382401:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829559.382405:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3445
00000100:00080000:1.0:1787829559.382409:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829559.382412:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829559.382418:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829559.382430:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4b40 x1874671626951552/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787829559.382441:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:1.0:1787829559.382441:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829559.382448:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829559.382451:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829559.382458:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3445
00000100:00080000:1.0:1787829559.382463:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829559.382468:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829559.382473:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829559.382483:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f4000 x1874671626951680/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829559.382488:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829559/real 1787829559]  req@ffff8cd0d48f7480 x1874671626951296/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829584 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829559.382508:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7480 x1874671626951296/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829584 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829559.382526:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829559.382529:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829559.382532:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626951296:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829559.382550:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626951424:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829559.382568:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829559.382578:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829559/real 1787829559]  req@ffff8cd0d48f6580 x1874671626951424/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829584 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829559.382586:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6580 x1874671626951424/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829584 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829559.382592:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829559.382594:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829559.382597:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626951424:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829559.382600:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626951552:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829559.382612:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829559.382622:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829559/real 1787829559]  req@ffff8cd0d48f4b40 x1874671626951552/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829584 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829559.382634:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4b40 x1874671626951552/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829584 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829559.382643:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829559.382659:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829559.382663:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626951552:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829559.382676:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626951680:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829559.382696:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829559.382705:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829559/real 1787829559]  req@ffff8cd0d48f4000 x1874671626951680/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829584 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829559.382714:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f4000 x1874671626951680/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829584 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829559.382719:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829559.382722:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829559.382724:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626951680:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829559.574252:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829559.958288:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829560.598176:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829560.982223:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829561.622211:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829562.006259:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829562.646164:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829563.030228:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829563.670505:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829564.054254:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787829564.502207:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829564.502219:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787829564.502223:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829564.502233:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3450
00000100:00080000:0.0:1787829564.502245:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829564.502252:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829564.502287:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c000 x1874671626951808/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829564.502317:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829564.502321:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829564.502323:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829564.502328:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3450
00000100:00100000:1.0:1787829564.502346:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626951808:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787829564.502347:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829564.502355:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829564.502370:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829564.502388:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f840 x1874671626951936/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829564.502401:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829564.502405:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787829564.502407:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829564.502411:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3450
00000100:00080000:0.0:1787829564.502416:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000800:00000100:1.0:1787829564.502417:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:0.0:1787829564.502424:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829564.502431:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829564.502444:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9f480 x1874671626952064/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829564.502455:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829564.502459:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829564.502462:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1787829564.502467:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829564/real 1787829564]  req@ffff8cd0c8e9c000 x1874671626951808/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829589 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829564.502468:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3450
00000100:00080000:0.0:1787829564.502477:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829564.502484:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829564.502490:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c000 x1874671626951808/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829589 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:0.0:1787829564.502491:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00080000:1.0:1787829564.502504:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829564.502509:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829564.502512:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cb40 x1874671626952192/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1787829564.502514:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626951808:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829564.502538:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626951936:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829564.502561:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829564.502574:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829564/real 1787829564]  req@ffff8cd0c8e9f840 x1874671626951936/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829589 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829564.502582:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f840 x1874671626951936/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829589 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829564.502588:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829564.502591:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829564.502593:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626951936:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787829564.502597:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626952064:192.168.203.145@tcp:8:kworker.0
00000800:00000100:1.0:1787829564.502608:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829564.502629:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829564/real 1787829564]  req@ffff8cd0c8e9f480 x1874671626952064/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829589 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829564.502650:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9f480 x1874671626952064/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829589 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829564.502656:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829564.502658:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829564.502660:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626952064:192.168.203.145@tcp:8:kworker.0
00000100:00100000:1.0:1787829564.502671:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626952192:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829564.502686:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829564.502700:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829564/real 1787829564]  req@ffff8cd0c8e9cb40 x1874671626952192/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829589 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829564.502708:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cb40 x1874671626952192/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829589 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829564.502713:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829564.502715:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829564.502717:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626952192:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829564.694175:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829565.078219:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829565.718148:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829566.102242:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829566.742145:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829567.126231:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829567.766146:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829568.150218:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829568.790138:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829569.174222:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829569.622209:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829569.622217:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829569.622220:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829569.622228:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3455
00000100:00080000:1.0:1787829569.622237:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829569.622242:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829569.622286:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626952320/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829569.622327:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829569.622331:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829569.622334:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829569.622338:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3455
00000100:00080000:1.0:1787829569.622342:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829569.622347:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829569.622358:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829569.622372:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626952448/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829569.622383:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829569.622387:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829569.622390:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829569.622394:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3455
00000100:00080000:1.0:1787829569.622398:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829569.622403:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829569.622409:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829569.622417:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626952320:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829569.622422:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626952576/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829569.622435:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829569.622437:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829569.622439:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829569.622442:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3455
00000100:00080000:1.0:1787829569.622451:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829569.622454:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829569.622459:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829569.622468:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626952704/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787829569.622491:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829569.622534:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829569/real 1787829569]  req@ffff8cd0d48f5a40 x1874671626952320/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829594 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829569.622548:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626952320/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829594 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829569.622556:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829569.622559:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829569.622562:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626952320:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829569.622567:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626952448:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829569.622601:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829569.622611:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829569/real 1787829569]  req@ffff8cd0d48f43c0 x1874671626952448/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829594 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829569.622620:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626952448/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829594 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829569.622626:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829569.622628:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829569.622630:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626952448:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829569.622659:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626952576:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829569.622674:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829569.622681:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829569/real 1787829569]  req@ffff8cd0d48f70c0 x1874671626952576/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829594 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829569.622689:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626952576/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829594 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829569.622694:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829569.622696:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829569.622698:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626952576:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829569.622702:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626952704:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829569.622711:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829569.622717:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829569/real 1787829569]  req@ffff8cd0d48f7840 x1874671626952704/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829594 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829569.622726:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626952704/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829594 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829569.622734:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829569.622738:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829569.622741:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626952704:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829569.814139:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829570.198218:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829570.838216:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829571.222181:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829571.862238:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829572.246244:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829572.886212:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829573.270224:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829573.910213:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829574.294296:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829574.742211:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829574.742226:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829574.742230:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829574.742238:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3460
00000100:00080000:2.0:1787829574.742248:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829574.742253:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829574.742291:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626952832/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829574.742312:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829574.742315:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829574.742317:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829574.742319:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3460
00000100:00080000:2.0:1787829574.742322:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829574.742324:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829574.742335:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829574.742348:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626952960/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829574.742361:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1787829574.742368:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626952832:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787829574.742380:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829574.742385:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829574.742392:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3460
00000100:00080000:2.0:1787829574.742396:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829574.742400:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829574.742406:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829574.742417:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626953088/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829574.742426:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829574.742428:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829574.742430:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829574.742432:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3460
00000100:00080000:2.0:1787829574.742436:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829574.742438:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829574.742441:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1787829574.742449:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787829574.742452:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626953216/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829574.742507:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829574/real 1787829574]  req@ffff8cd0c3dab0c0 x1874671626952832/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829599 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829574.742530:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626952832/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829599 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829574.742560:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829574.742565:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829574.742569:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626952832:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829574.742594:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626952960:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829574.742622:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829574.742637:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829574/real 1787829574]  req@ffff8cd0c3da9a40 x1874671626952960/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829599 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829574.742652:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626952960/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829599 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829574.742661:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829574.742665:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829574.742669:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626952960:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829574.742676:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626953088:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829574.742698:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829574.742711:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829574/real 1787829574]  req@ffff8cd0c3da8780 x1874671626953088/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829599 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829574.742749:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626953088/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829599 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829574.742758:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829574.742762:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829574.742765:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626953088:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829574.742773:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626953216:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829574.742797:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829574.742810:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829574/real 1787829574]  req@ffff8cd0c3daa940 x1874671626953216/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829599 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829574.742824:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626953216/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829599 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829574.742833:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829574.742836:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829574.742839:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626953216:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829574.934162:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829575.318229:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829575.958189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829576.342299:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829576.982221:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829577.366248:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829578.006162:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829578.390366:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829579.030224:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829579.414207:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829579.862174:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829579.862182:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829579.862185:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829579.862193:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3465
00000100:00080000:2.0:1787829579.862203:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829579.862208:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829579.862244:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626953344/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829579.862287:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829579.862290:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829579.862291:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829579.862294:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3465
00000100:00080000:2.0:1787829579.862297:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829579.862301:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829579.862309:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829579.862320:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8b40 x1874671626953472/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829579.862327:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626953344:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787829579.862331:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829579.862335:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829579.862337:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829579.862340:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3465
00000100:00080000:2.0:1787829579.862342:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829579.862345:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829579.862349:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829579.862358:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626953600/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829579.862365:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829579.862368:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829579.862369:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829579.862371:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3465
00000100:00080000:2.0:1787829579.862380:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829579.862384:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829579.862387:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829579.862395:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626953728/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829579.862428:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626953472:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829579.862465:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626953600:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829579.862519:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626953728:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829580.054210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829580.438304:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829581.078176:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829581.462251:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829582.102212:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829582.486293:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:1.0:1787829582.934455:0:2344:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:1.0:1787829582.934678:0:2344:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:1.0:1787829582.934700:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787829582.934753:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787829582.934762:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:1.0:1787829582.934769:0:2344:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:0.0:1787829582.935496:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829579/real 1787829582]  req@ffff8cd0c3da8b40 x1874671626953472/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829604 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829582.935530:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8b40 x1874671626953472/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829604 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829582.935542:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829582.935548:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829582.935552:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626953472:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829582.935560:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829579/real 1787829582]  req@ffff8cd0c3da83c0 x1874671626953600/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829604 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829582.935572:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626953600/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829604 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829582.935580:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829582.935584:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829582.935588:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626953600:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829582.935597:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829579/real 1787829582]  req@ffff8cd0c3da9680 x1874671626953728/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829604 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829582.935611:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626953728/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829604 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829582.935620:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829582.935644:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829582.935649:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626953728:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829582.935660:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829579/real 1787829582]  req@ffff8cd0c3da8f00 x1874671626953344/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829604 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829582.935677:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626953344/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829604 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829582.935701:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829582.935706:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829582.935710:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626953344:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829583.126203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829583.510203:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829584.150418:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829584.534277:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829584.982201:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829584.982209:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829584.982212:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829584.982221:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3470
00000100:00080000:2.0:1787829584.982230:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829584.982235:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829584.982267:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626953856/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829584.982290:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829584.982293:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829584.982294:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829584.982297:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3470
00000100:00080000:2.0:1787829584.982299:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829584.982301:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829584.982309:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829584.982317:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626953984/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829584.982326:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1787829584.982337:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626953856:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787829584.982346:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829584.982348:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829584.982352:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3470
00000100:00080000:2.0:1787829584.982355:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829584.982358:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829584.982363:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829584.982373:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626954112/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829584.982381:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829584.982384:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829584.982386:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829584.982389:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3470
00000100:00080000:2.0:1787829584.982391:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829584.982395:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:0.0:1787829584.982399:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:2.0:1787829584.982401:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829584.982411:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8b40 x1874671626954240/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829584.982435:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829584/real 1787829584]  req@ffff8cd0c3da8f00 x1874671626953856/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829609 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829584.982450:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626953856/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829609 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829584.982458:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829584.982462:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829584.982466:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626953856:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829584.982492:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626953984:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829584.982516:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829584.982530:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829584/real 1787829584]  req@ffff8cd0c3da9680 x1874671626953984/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829609 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829584.982553:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626953984/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829609 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829584.982561:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829584.982564:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829584.982568:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626953984:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829584.982591:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626954112:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829584.982614:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829584.982625:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829584/real 1787829584]  req@ffff8cd0c3da83c0 x1874671626954112/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829609 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829584.982637:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626954112/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829609 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829584.982644:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829584.982648:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829584.982652:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626954112:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829584.982658:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626954240:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829584.982683:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829584.982695:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829584/real 1787829584]  req@ffff8cd0c3da8b40 x1874671626954240/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829609 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829584.982706:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8b40 x1874671626954240/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829609 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829584.982727:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829584.982731:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829584.982734:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626954240:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829585.174192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829585.558283:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829586.198181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829586.582269:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829587.222165:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829587.606194:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829588.246206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829588.630280:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829589.270210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829589.654257:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829590.102181:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829590.102192:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829590.102196:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829590.102204:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3475
00000100:00080000:1.0:1787829590.102215:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829590.102221:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829590.102261:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626954368/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829590.102290:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829590.102293:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829590.102294:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829590.102297:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3475
00000100:00080000:1.0:1787829590.102300:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829590.102318:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829590.102327:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829590.102339:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626954496/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829590.102343:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626954368:192.168.203.145@tcp:250:kworker.0
00000100:00080000:1.0:1787829590.102351:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829590.102355:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829590.102356:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829590.102359:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3475
00000100:00080000:1.0:1787829590.102362:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829590.102365:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829590.102368:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829590.102377:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626954624/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829590.102384:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829590.102387:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829590.102388:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829590.102390:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3475
00000100:00080000:1.0:1787829590.102400:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829590.102403:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829590.102406:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1787829590.102411:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829590.102417:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626954752/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829590.102476:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829590/real 1787829590]  req@ffff8cd0d48f61c0 x1874671626954368/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829615 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829590.102498:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626954368/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829615 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829590.102510:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829590.102514:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829590.102519:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626954368:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829590.102542:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626954496:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829590.102567:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829590.102581:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829590/real 1787829590]  req@ffff8cd0d48f6d00 x1874671626954496/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829615 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829590.102594:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626954496/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829615 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829590.102617:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829590.102620:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829590.102624:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626954496:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829590.102631:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626954624:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829590.102656:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829590.102667:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829590/real 1787829590]  req@ffff8cd0d48f52c0 x1874671626954624/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829615 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829590.102681:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626954624/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829615 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829590.102689:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829590.102693:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829590.102696:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626954624:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829590.102703:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626954752:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829590.102723:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829590.102735:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829590/real 1787829590]  req@ffff8cd0d48f7c00 x1874671626954752/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829615 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829590.102748:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626954752/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829615 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829590.102757:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829590.102760:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829590.102769:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626954752:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829590.294194:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829590.678292:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829591.318178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829591.702223:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829592.342627:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:3.0:1787829592.726274:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829593.366188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829593.750238:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829594.390246:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829594.774262:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829595.222167:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829595.222177:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829595.222181:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829595.222193:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3481
00000100:00080000:2.0:1787829595.222204:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829595.222211:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829595.222248:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626954880/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829595.222300:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829595.222305:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829595.222310:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829595.222315:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3481
00000100:00080000:2.0:1787829595.222321:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829595.222326:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829595.222336:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829595.222339:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626954880:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787829595.222354:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626955008/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829595.222368:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829595.222373:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829595.222375:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829595.222379:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3481
00000100:00080000:2.0:1787829595.222384:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829595.222389:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:0.0:1787829595.222395:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:2.0:1787829595.222395:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829595.222432:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626955136/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829595.222443:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829595.222447:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829595.222449:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829595.222454:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3481
00000100:00100000:0.0:1787829595.222461:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829595/real 1787829595]  req@ffff8cd0c3daa940 x1874671626954880/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829620 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829595.222468:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829595.222480:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829595.222486:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626954880/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829620 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1787829595.222487:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1787829595.222502:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829595.222506:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829595.222512:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626954880:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787829595.222513:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626955264/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829595.222533:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626955008:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829595.222556:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829595.222565:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829595/real 1787829595]  req@ffff8cd0c3da8780 x1874671626955008/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829620 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829595.222574:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626955008/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829620 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829595.222580:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829595.222582:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829595.222585:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626955008:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829595.222597:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626955136:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829595.222612:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829595.222620:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829595/real 1787829595]  req@ffff8cd0c3da9a40 x1874671626955136/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829620 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829595.222628:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626955136/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829620 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829595.222633:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829595.222635:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829595.222637:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626955136:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829595.222641:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626955264:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829595.222651:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829595.222656:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829595/real 1787829595]  req@ffff8cd0c3dab0c0 x1874671626955264/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829620 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829595.222663:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626955264/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829620 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829595.222668:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829595.222670:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829595.222672:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626955264:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829595.414178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829595.798274:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829596.438164:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829596.822235:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829597.462255:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829597.846223:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829598.486220:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829598.870241:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829599.510205:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829599.894247:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829600.342230:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829600.342270:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829600.342274:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829600.342285:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3486
00000100:00080000:1.0:1787829600.342296:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829600.342304:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829600.342341:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7c00 x1874671626955392/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829600.342381:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829600.342385:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829600.342388:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829600.342392:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3486
00000100:00080000:1.0:1787829600.342396:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829600.342402:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829600.342412:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829600.342424:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626955392:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829600.342429:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f52c0 x1874671626955520/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829600.342443:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829600.342448:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829600.342450:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829600.342454:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3486
00000100:00080000:1.0:1787829600.342471:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829600.342475:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:0.0:1787829600.342479:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:1.0:1787829600.342483:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829600.342499:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f6d00 x1874671626955648/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829600.342516:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:0.0:1787829600.342517:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829600/real 1787829600]  req@ffff8cd0d48f7c00 x1874671626955392/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829625 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829600.342527:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829600.342533:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829600.342540:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3486
00000100:00100000:0.0:1787829600.342543:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7c00 x1874671626955392/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829625 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829600.342547:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829600.342556:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00080000:0.0:1787829600.342560:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000008:00100000:1.0:1787829600.342564:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1787829600.342568:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829600.342574:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626955392:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829600.342582:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f61c0 x1874671626955776/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829600.342602:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626955520:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829600.342627:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829600.342637:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829600/real 1787829600]  req@ffff8cd0d48f52c0 x1874671626955520/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829625 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829600.342646:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f52c0 x1874671626955520/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829625 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829600.342652:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829600.342654:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829600.342656:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626955520:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829600.342660:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626955648:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829600.342672:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829600.342678:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829600/real 1787829600]  req@ffff8cd0d48f6d00 x1874671626955648/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829625 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829600.342694:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f6d00 x1874671626955648/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829625 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829600.342699:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829600.342702:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829600.342704:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626955648:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829600.342708:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626955776:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829600.342721:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829600.342727:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829600/real 1787829600]  req@ffff8cd0d48f61c0 x1874671626955776/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829625 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829600.342735:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f61c0 x1874671626955776/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829625 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829600.342740:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829600.342741:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829600.342743:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626955776:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829600.534183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829600.918237:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829601.558173:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829601.942176:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829602.582189:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829602.966247:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829603.606213:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829603.990192:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829604.630181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829605.014223:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787829605.462166:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829605.462174:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787829605.462178:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829605.462200:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3491
00000100:00080000:0.0:1787829605.462208:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829605.462213:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829605.462242:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d2c0 x1874671626955904/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829605.462266:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829605.462270:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829605.462272:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829605.462276:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3491
00000100:00080000:0.0:1787829605.462280:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829605.462284:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829605.462293:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829605.462298:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626955904:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829605.462306:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626956032/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829605.462316:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829605.462320:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787829605.462321:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829605.462338:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3491
00000100:00080000:0.0:1787829605.462356:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829605.462360:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829605.462366:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:1.0:1787829605.462366:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829605.462376:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626956160/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829605.462397:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829605.462400:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829605.462402:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1787829605.462404:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829605/real 1787829605]  req@ffff8cd0c8e9d2c0 x1874671626955904/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829630 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829605.462406:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3491
00000100:00080000:0.0:1787829605.462410:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829605.462414:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829605.462419:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829605.462420:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d2c0 x1874671626955904/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829630 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829605.462428:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626956288/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829605.462431:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829605.462435:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829605.462439:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626955904:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829605.462462:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626956032:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829605.462485:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829605.462496:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829605/real 1787829605]  req@ffff8cd0c8e9d680 x1874671626956032/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829630 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829605.462510:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626956032/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829630 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829605.462516:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829605.462518:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829605.462521:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626956032:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787829605.462525:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626956160:192.168.203.145@tcp:8:kworker.0
00000800:00000100:1.0:1787829605.462539:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829605.462546:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829605/real 1787829605]  req@ffff8cd0c8e9c3c0 x1874671626956160/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829630 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829605.462553:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626956160/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829630 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829605.462558:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829605.462560:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829605.462562:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626956160:192.168.203.145@tcp:8:kworker.0
00000100:00100000:1.0:1787829605.462565:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626956288:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829605.462575:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829605.462581:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829605/real 1787829605]  req@ffff8cd0c8e9e940 x1874671626956288/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829630 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829605.462588:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626956288/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829630 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829605.462607:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829605.462611:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829605.462614:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626956288:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829605.654203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829606.038213:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829606.678204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829607.062196:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829607.702183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829608.086205:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829608.726192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829609.110183:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829609.750170:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829610.134203:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:1.0:1787829610.582221:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829610.582232:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:1.0:1787829610.582237:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829610.582247:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3496
00000100:00080000:1.0:1787829610.582259:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829610.582266:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829610.582314:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f7840 x1874671626956416/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829610.582336:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829610.582339:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829610.582341:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829610.582343:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3496
00000100:00080000:1.0:1787829610.582347:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829610.582350:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829610.582359:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829610.582371:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626956416:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829610.582372:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f70c0 x1874671626956544/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829610.582386:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829610.582388:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:1.0:1787829610.582402:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829610.582406:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3496
00000100:00080000:1.0:1787829610.582429:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829610.582432:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829610.582437:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1787829610.582438:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829610.582451:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f43c0 x1874671626956672/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829610.582459:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:1.0:1787829610.582462:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:1.0:1787829610.582463:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:1.0:1787829610.582466:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3496
00000100:00080000:1.0:1787829610.582468:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829610.582471:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:1.0:1787829610.582475:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829610.582476:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829610/real 1787829610]  req@ffff8cd0d48f7840 x1874671626956416/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829635 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829610.582486:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0d48f5a40 x1874671626956800/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829610.582503:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f7840 x1874671626956416/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829635 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829610.582516:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829610.582521:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829610.582525:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626956416:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829610.582557:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626956544:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829610.582582:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829610.582594:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829610/real 1787829610]  req@ffff8cd0d48f70c0 x1874671626956544/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829635 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829610.582604:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f70c0 x1874671626956544/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829635 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829610.582610:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829610.582612:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829610.582615:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626956544:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829610.582619:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626956672:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829610.582633:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829610.582647:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829610/real 1787829610]  req@ffff8cd0d48f43c0 x1874671626956672/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829635 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829610.582655:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f43c0 x1874671626956672/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829635 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829610.582664:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829610.582667:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829610.582671:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626956672:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829610.582677:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626956800:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829610.582697:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829610.582708:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829610/real 1787829610]  req@ffff8cd0d48f5a40 x1874671626956800/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829635 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829610.582740:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0d48f5a40 x1874671626956800/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829635 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829610.582747:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829610.582750:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829610.582753:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0d48f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626956800:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829610.774166:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829611.158157:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829611.798185:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829612.182204:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829612.822187:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829613.206209:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829613.846160:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829614.230170:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829614.870180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829615.254228:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829615.702188:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829615.702198:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829615.702203:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829615.702212:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3501
00000100:00080000:2.0:1787829615.702224:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829615.702232:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829615.702285:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626956928/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829615.702315:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829615.702320:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829615.702322:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829615.702326:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3501
00000100:00080000:2.0:1787829615.702331:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829615.702336:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829615.702347:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829615.702358:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626956928:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787829615.702364:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626957056/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829615.702380:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829615.702396:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829615.702399:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829615.702404:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3501
00000100:00080000:2.0:1787829615.702409:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829615.702414:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829615.702420:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1787829615.702421:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787829615.702439:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626957184/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829615.702451:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829615.702454:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:0.0:1787829615.702467:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829615/real 1787829615]  req@ffff8cd0c3dabc00 x1874671626956928/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829640 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829615.702468:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829615.702477:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3501
00000100:00080000:2.0:1787829615.702485:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829615.702492:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829615.702498:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626956928/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829640 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:2.0:1787829615.702498:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00080000:0.0:1787829615.702516:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:2.0:1787829615.702518:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626957312/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829615.702535:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829615.702541:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626956928:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829615.702568:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626957056:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829615.702595:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829615.702607:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829615/real 1787829615]  req@ffff8cd0c3daa580 x1874671626957056/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829640 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829615.702623:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626957056/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829640 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829615.702629:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829615.702632:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829615.702634:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626957056:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829615.702638:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626957184:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829615.702654:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829615.702662:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829615/real 1787829615]  req@ffff8cd0c3dab840 x1874671626957184/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829640 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829615.702670:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626957184/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829640 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829615.702675:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829615.702677:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829615.702679:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626957184:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829615.702683:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626957312:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829615.702693:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829615.702699:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829615/real 1787829615]  req@ffff8cd0c3da8000 x1874671626957312/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829640 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829615.702706:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626957312/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829640 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829615.702728:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829615.702730:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829615.702732:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626957312:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829615.894158:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829616.278202:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829616.918178:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829617.302190:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829617.942183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829618.326213:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829618.966158:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829619.350209:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829619.990201:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829620.374192:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829620.822156:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829620.822164:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829620.822168:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829620.822176:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3506
00000100:00080000:2.0:1787829620.822184:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829620.822190:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829620.822235:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626957440/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829620.822268:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829620.822271:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829620.822272:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829620.822275:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3506
00000100:00080000:2.0:1787829620.822278:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829620.822281:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829620.822289:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829620.822299:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626957568/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829620.822308:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829620.822311:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829620.822314:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:0.0:1787829620.822315:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626957440:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787829620.822319:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3506
00000100:00080000:2.0:1787829620.822324:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829620.822328:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829620.822331:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829620.822339:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626957696/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829620.822346:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829620.822348:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829620.822349:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829620.822352:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3506
00000100:00080000:2.0:1787829620.822360:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829620.822363:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829620.822368:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:0.0:1787829620.822377:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:2.0:1787829620.822378:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daad00 x1874671626957824/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829620.822419:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829620/real 1787829620]  req@ffff8cd0c3da9e00 x1874671626957440/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829645 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829620.822437:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626957440/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829645 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829620.822446:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829620.822450:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829620.822454:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626957440:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829620.822472:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626957568:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829620.822487:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829620.822496:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829620/real 1787829620]  req@ffff8cd0c3da92c0 x1874671626957568/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829645 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829620.822504:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626957568/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829645 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829620.822515:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829620.822518:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829620.822520:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626957568:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829620.822525:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626957696:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829620.822540:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829620.822546:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829620/real 1787829620]  req@ffff8cd0c3daa1c0 x1874671626957696/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829645 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829620.822554:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626957696/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829645 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829620.822559:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829620.822561:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829620.822563:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626957696:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829620.822567:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626957824:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829620.822577:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829620.822583:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829620/real 1787829620]  req@ffff8cd0c3daad00 x1874671626957824/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829645 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829620.822591:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daad00 x1874671626957824/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829645 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829620.822596:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829620.822597:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829620.822603:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626957824:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829621.014421:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829621.398209:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829622.038169:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829622.422192:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829623.062176:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829623.446165:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829624.086228:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829624.470217:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829625.110169:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829625.494214:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829625.942215:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829625.942224:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829625.942239:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829625.942248:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3511
00000100:00080000:2.0:1787829625.942258:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829625.942264:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829625.942311:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daad00 x1874671626957952/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829625.942344:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829625.942349:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829625.942351:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829625.942355:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3511
00000100:00080000:2.0:1787829625.942360:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829625.942376:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829625.942387:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829625.942403:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa1c0 x1874671626958080/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829625.942417:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829625.942435:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829625.942437:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829625.942441:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3511
00000100:00080000:2.0:1787829625.942461:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1787829625.942469:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626957952:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787829625.942471:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829625.942483:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829625.942495:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da92c0 x1874671626958208/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829625.942507:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829625.942511:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829625.942512:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829625.942515:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3511
00000100:00080000:2.0:1787829625.942518:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829625.942520:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829625.942524:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829625.942532:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9e00 x1874671626958336/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787829625.942563:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829625.942613:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829625/real 1787829625]  req@ffff8cd0c3daad00 x1874671626957952/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829650 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829625.942629:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daad00 x1874671626957952/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829650 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829625.942641:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829625.942646:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829625.942651:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626957952:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829625.942660:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626958080:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829625.942688:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829625.942702:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829625/real 1787829625]  req@ffff8cd0c3daa1c0 x1874671626958080/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829650 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829625.942712:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa1c0 x1874671626958080/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829650 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829625.942718:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829625.942721:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829625.942735:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626958080:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829625.942762:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626958208:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829625.942780:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829625.942788:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829625/real 1787829625]  req@ffff8cd0c3da92c0 x1874671626958208/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829650 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829625.942796:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da92c0 x1874671626958208/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829650 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829625.942801:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829625.942803:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829625.942805:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626958208:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829625.942809:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626958336:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829625.942827:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829625.942839:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829625/real 1787829625]  req@ffff8cd0c3da9e00 x1874671626958336/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829650 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829625.942851:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9e00 x1874671626958336/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829650 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829625.942858:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829625.942862:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829625.942865:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626958336:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829626.134239:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829626.518177:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829627.158146:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829627.542251:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829628.182193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829628.566190:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829629.206149:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829629.590258:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829630.230188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829630.614176:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787829631.062197:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829631.062209:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787829631.062213:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829631.062223:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3516
00000100:00080000:3.0:1787829631.062235:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829631.062242:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787829631.062281:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626958464/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829631.062308:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829631.062313:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787829631.062315:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829631.062321:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3516
00000100:00080000:3.0:1787829631.062326:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:0.0:1787829631.062337:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626958464:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787829631.062358:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829631.062371:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829631.062388:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626958592/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787829631.062397:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787829631.062408:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829631.062421:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787829631.062424:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829631.062430:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3516
00000100:00080000:3.0:1787829631.062435:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829631.062439:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829631.062445:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829631.062457:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829631/real 1787829631]  req@ffff8cd0c8ead2c0 x1874671626958464/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829656 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:3.0:1787829631.062460:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626958720/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829631.062488:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829631.062494:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00100000:0.0:1787829631.062495:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626958464/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829656 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:3.0:1787829631.062499:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829631.062509:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:3.0:1787829631.062510:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3516
00000100:00080000:0.0:1787829631.062514:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829631.062518:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626958464:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787829631.062518:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829631.062528:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829631.062534:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829631.062542:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626958592:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829631.062550:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626958848/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787829631.062565:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829631.062578:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829631/real 1787829631]  req@ffff8cd0c8eae1c0 x1874671626958592/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829656 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829631.062586:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626958592/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829656 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829631.062592:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829631.062594:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829631.062597:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626958592:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829631.062600:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626958720:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829631.062622:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829631.062629:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829631/real 1787829631]  req@ffff8cd0c8eada40 x1874671626958720/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829656 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829631.062637:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626958720/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829656 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829631.062642:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829631.062644:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829631.062646:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626958720:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829631.062656:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626958848:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829631.062666:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829631.062674:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829631/real 1787829631]  req@ffff8cd0c8eac780 x1874671626958848/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829656 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829631.062687:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626958848/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829656 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829631.062695:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829631.062698:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829631.062701:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626958848:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829631.254125:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829631.638256:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829632.278135:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829632.662229:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829633.302154:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829633.686247:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829634.326188:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829634.710212:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829635.350159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829635.734204:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787829636.182174:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829636.182182:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787829636.182186:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829636.182193:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3522
00000100:00080000:3.0:1787829636.182202:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829636.182207:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787829636.182239:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626958976/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829636.182262:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829636.182265:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787829636.182267:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829636.182269:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3522
00000100:00080000:3.0:1787829636.182271:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829636.182274:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829636.182281:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829636.182290:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626959104/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829636.182298:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829636.182301:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787829636.182302:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829636.182305:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3522
00000100:00100000:0.0:1787829636.182314:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626958976:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787829636.182321:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829636.182326:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829636.182334:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829636.182344:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626959232/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829636.182352:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829636.182355:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787829636.182357:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829636.182360:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3522
00000100:00080000:3.0:1787829636.182362:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829636.182371:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829636.182374:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829636.182382:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626959360/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787829636.182392:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829636.182447:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829636/real 1787829636]  req@ffff8cd0c8eaf0c0 x1874671626958976/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829661 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829636.182466:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626958976/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829661 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829636.182477:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829636.182482:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829636.182487:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626958976:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829636.182494:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626959104:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829636.182518:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829636.182532:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829636/real 1787829636]  req@ffff8cd0c8eac3c0 x1874671626959104/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829661 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829636.182544:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626959104/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829661 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829636.182553:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829636.182558:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829636.182561:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626959104:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829636.182585:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626959232:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829636.182604:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829636.182616:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829636/real 1787829636]  req@ffff8cd0c8eac780 x1874671626959232/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829661 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829636.182641:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626959232/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829661 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829636.182649:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829636.182652:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829636.182655:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626959232:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829636.182662:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626959360:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829636.182683:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829636.182695:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829636/real 1787829636]  req@ffff8cd0c8eada40 x1874671626959360/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829661 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829636.182707:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626959360/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829661 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829636.182715:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829636.182719:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829636.182722:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626959360:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829636.374137:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829636.758201:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829637.398183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829637.782214:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829638.422181:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829638.806202:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829639.446186:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829639.830225:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829640.470173:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829640.854211:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829641.302171:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829641.302179:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829641.302183:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829641.302191:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3527
00000100:00080000:2.0:1787829641.302200:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829641.302206:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829641.302245:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626959488/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829641.302281:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829641.302283:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829641.302285:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829641.302288:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3527
00000100:00080000:2.0:1787829641.302291:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829641.302297:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829641.302305:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829641.302316:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626959616/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829641.302324:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829641.302327:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829641.302329:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829641.302331:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3527
00000100:00080000:2.0:1787829641.302334:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829641.302338:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829641.302341:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626959488:192.168.203.145@tcp:250:kworker.0
00000008:00100000:2.0:1787829641.302350:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829641.302366:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626959744/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829641.302378:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829641.302382:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829641.302384:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1787829641.302397:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787829641.302401:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3527
00000100:00080000:2.0:1787829641.302412:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829641.302416:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829641.302421:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829641.302432:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626959872/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829641.302449:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829641/real 1787829641]  req@ffff8cd0c3da8000 x1874671626959488/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829666 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829641.302470:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626959488/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829666 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829641.302478:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829641.302481:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829641.302484:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626959488:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829641.302489:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626959616:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829641.302507:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829641.302518:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829641/real 1787829641]  req@ffff8cd0c3dab840 x1874671626959616/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829666 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829641.302526:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626959616/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829666 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829641.302532:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829641.302534:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829641.302536:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626959616:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829641.302554:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626959744:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829641.302572:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829641.302582:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829641/real 1787829641]  req@ffff8cd0c3daa580 x1874671626959744/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829666 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829641.302595:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626959744/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829666 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829641.302616:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829641.302619:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829641.302622:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626959744:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829641.302626:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626959872:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829641.302642:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829641.302650:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829641/real 1787829641]  req@ffff8cd0c3dabc00 x1874671626959872/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829666 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829641.302658:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626959872/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829666 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829641.302663:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829641.302665:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829641.302667:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626959872:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829641.494595:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829641.878215:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829642.518186:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829642.902216:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829643.542167:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829643.926222:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829644.566187:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829644.950265:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829645.590222:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829645.974239:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787829646.422187:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829646.422195:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787829646.422215:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829646.422223:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3532
00000100:00080000:3.0:1787829646.422232:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829646.422237:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787829646.422267:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626960000/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829646.422290:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829646.422293:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787829646.422295:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829646.422298:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3532
00000100:00080000:3.0:1787829646.422300:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829646.422302:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829646.422310:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829646.422320:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626960128/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829646.422329:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829646.422332:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787829646.422333:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829646.422336:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3532
00000100:00080000:3.0:1787829646.422337:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829646.422339:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829646.422344:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829646.422345:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626960000:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829646.422354:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626960256/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829646.422372:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829646.422375:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787829646.422378:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829646.422383:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3532
00000100:00080000:3.0:1787829646.422395:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829646.422400:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829646.422406:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829646.422421:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626960384/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829646.422457:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626960128:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829646.422497:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626960256:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829646.422523:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626960384:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829646.614184:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829646.998276:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829647.638225:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829648.022301:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829648.662180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829649.046228:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000400:00000100:2.0:1787829649.494369:0:2346:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.145:988
00000400:00000100:2.0:1787829649.494414:0:2346:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.145@tcp at host 192.168.203.145 was unreachable: the network or that node may be down, or Lustre may be misconfigured.
00000800:00000100:2.0:1787829649.494428:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787829649.494462:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787829649.494473:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000800:00000100:2.0:1787829649.494481:0:2346:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.145@tcp
00000100:00100000:0.0:1787829649.494585:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829646/real 1787829649]  req@ffff8cd0c8eac000 x1874671626960256/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829671 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829649.494611:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626960256/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829671 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829649.494624:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829649.494629:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829649.494634:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626960256:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829649.494642:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829646/real 1787829649]  req@ffff8cd0c8eae940 x1874671626960384/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829671 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829649.494655:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626960384/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829671 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829649.494663:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829649.494666:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829649.494670:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626960384:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829649.494677:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829646/real 1787829649]  req@ffff8cd0c8eae1c0 x1874671626960000/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829671 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829649.494689:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626960000/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829671 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829649.494698:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829649.494714:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829649.494718:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626960000:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829649.494726:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829646/real 1787829649]  req@ffff8cd0c8ead2c0 x1874671626960128/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829671 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829649.494739:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626960128/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829671 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829649.494748:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829649.494752:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829649.494755:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626960128:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829649.686193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829650.070166:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829650.710173:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829651.094197:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829651.542234:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829651.542245:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829651.542249:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829651.542259:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3537
00000100:00080000:2.0:1787829651.542270:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829651.542278:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829651.542311:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626960512/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829651.542335:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829651.542338:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829651.542341:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829651.542345:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3537
00000100:00080000:2.0:1787829651.542349:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829651.542370:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829651.542380:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829651.542396:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626960512:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787829651.542398:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626960640/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829651.542417:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829651.542421:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829651.542423:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829651.542427:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3537
00000100:00080000:2.0:1787829651.542432:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829651.542436:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829651.542442:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829651.542456:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626960768/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829651.542464:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829651.542467:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829651.542469:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829651.542473:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3537
00000800:00000100:0.0:1787829651.542480:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787829651.542480:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829651.542490:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829651.542496:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829651.542510:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626960896/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829651.542539:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829651/real 1787829651]  req@ffff8cd0c3dab0c0 x1874671626960512/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829676 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829651.542560:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626960512/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829676 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829651.542571:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829651.542576:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829651.542581:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626960512:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829651.542621:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626960640:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829651.542646:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829651.542669:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829651/real 1787829651]  req@ffff8cd0c3da9a40 x1874671626960640/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829676 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829651.542684:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626960640/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829676 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829651.542693:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829651.542697:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829651.542701:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626960640:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829651.542707:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626960768:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829651.542746:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829651.542760:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829651/real 1787829651]  req@ffff8cd0c3da8780 x1874671626960768/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829676 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829651.542773:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626960768/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829676 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829651.542780:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829651.542783:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829651.542786:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626960768:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829651.542792:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626960896:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829651.542813:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829651.542824:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829651/real 1787829651]  req@ffff8cd0c3daa940 x1874671626960896/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829676 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829651.542835:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626960896/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829676 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829651.542850:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829651.542854:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829651.542857:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626960896:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829651.734203:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829652.118178:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829652.758213:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829653.142244:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829653.782180:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829654.166207:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829654.806183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829655.190253:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829655.830224:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829656.214200:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829656.662214:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829656.662248:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829656.662253:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829656.662265:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3542
00000100:00080000:2.0:1787829656.662278:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829656.662286:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829656.662326:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8b40 x1874671626961024/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829656.662349:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829656.662352:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829656.662354:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829656.662357:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3542
00000100:00080000:2.0:1787829656.662359:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829656.662363:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829656.662380:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829656.662381:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626961024:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787829656.662394:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626961152/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829656.662404:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829656.662408:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829656.662410:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829656.662414:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3542
00000100:00080000:2.0:1787829656.662423:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829656.662428:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829656.662435:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829656.662448:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626961280/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:0.0:1787829656.662466:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787829656.662468:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829656.662473:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829656.662476:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829656.662482:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3542
00000100:00080000:2.0:1787829656.662488:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829656.662494:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829656.662500:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829656.662511:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626961408/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829656.662532:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829656/real 1787829656]  req@ffff8cd0c3da8b40 x1874671626961024/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829681 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829656.662550:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8b40 x1874671626961024/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829681 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829656.662559:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829656.662563:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829656.662567:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626961024:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829656.662591:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626961152:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829656.662610:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829656.662619:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829656/real 1787829656]  req@ffff8cd0c3da83c0 x1874671626961152/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829681 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829656.662627:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626961152/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829681 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829656.662640:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829656.662643:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829656.662645:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626961152:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829656.662650:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626961280:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829656.662666:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829656.662674:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829656/real 1787829656]  req@ffff8cd0c3da9680 x1874671626961280/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829681 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829656.662681:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626961280/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829681 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829656.662687:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829656.662689:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829656.662691:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626961280:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829656.662695:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626961408:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829656.662705:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829656.662711:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829656/real 1787829656]  req@ffff8cd0c3da8f00 x1874671626961408/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829681 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829656.662718:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626961408/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829681 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829656.662724:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829656.662726:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829656.662732:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626961408:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829656.854265:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829657.238230:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829657.878208:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829658.262221:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829658.902195:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829659.286213:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829659.926152:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829660.310189:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829660.950221:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829661.334206:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829661.782185:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829661.782193:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829661.782196:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829661.782204:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3547
00000100:00080000:2.0:1787829661.782213:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829661.782225:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829661.782260:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8f00 x1874671626961536/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829661.782282:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829661.782285:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829661.782286:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829661.782289:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3547
00000100:00080000:2.0:1787829661.782291:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829661.782294:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829661.782313:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829661.782323:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626961536:192.168.203.145@tcp:250:kworker.0
00000100:00100000:2.0:1787829661.782326:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9680 x1874671626961664/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829661.782337:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829661.782340:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829661.782341:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829661.782345:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3547
00000100:00080000:2.0:1787829661.782348:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829661.782350:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829661.782354:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829661.782365:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da83c0 x1874671626961792/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829661.782375:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829661.782379:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829661.782381:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000800:00000100:0.0:1787829661.782386:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787829661.782388:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3547
00000100:00080000:2.0:1787829661.782394:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829661.782400:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829661.782405:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829661.782418:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8b40 x1874671626961920/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:0.0:1787829661.782436:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829661/real 1787829661]  req@ffff8cd0c3da8f00 x1874671626961536/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829686 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829661.782468:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8f00 x1874671626961536/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829686 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829661.782479:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829661.782483:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829661.782486:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626961536:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829661.782510:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626961664:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829661.782536:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829661.782550:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829661/real 1787829661]  req@ffff8cd0c3da9680 x1874671626961664/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829686 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829661.782560:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9680 x1874671626961664/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829686 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829661.782566:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829661.782568:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829661.782578:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626961664:192.168.203.145@tcp:38:kworker.0
00000100:00100000:0.0:1787829661.782582:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626961792:192.168.203.145@tcp:8:kworker.0
00000800:00000100:0.0:1787829661.782599:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829661.782607:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829661/real 1787829661]  req@ffff8cd0c3da83c0 x1874671626961792/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829686 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829661.782615:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da83c0 x1874671626961792/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829686 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829661.782621:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829661.782623:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829661.782632:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626961792:192.168.203.145@tcp:8:kworker.0
00000100:00100000:0.0:1787829661.782636:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626961920:192.168.203.145@tcp:38:kworker.0
00000800:00000100:0.0:1787829661.782649:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829661.782659:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829661/real 1787829661]  req@ffff8cd0c3da8b40 x1874671626961920/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829686 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:0.0:1787829661.782681:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8b40 x1874671626961920/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829686 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829661.782690:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:0.0:1787829661.782694:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:0.0:1787829661.782697:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626961920:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829661.974279:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829662.358163:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829662.998152:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829663.382238:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829664.022139:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829664.406191:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829665.046234:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829665.430244:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829666.070179:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829666.454246:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787829666.902287:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829666.902301:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787829666.902306:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829666.902319:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3552
00000100:00080000:0.0:1787829666.902333:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829666.902343:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829666.902396:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626962048/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829666.902434:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829666.902438:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829666.902440:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829666.902445:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3552
00000100:00080000:0.0:1787829666.902449:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829666.902454:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829666.902466:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829666.902484:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626962176/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1787829666.902493:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626962048:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787829666.902516:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829666.902520:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787829666.902522:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829666.902529:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3552
00000100:00080000:0.0:1787829666.902533:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829666.902557:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000800:00000100:1.0:1787829666.902563:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000008:00100000:0.0:1787829666.902568:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829666.902590:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626962304/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829666.902603:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829666.902607:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829666.902609:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1787829666.902611:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829666/real 1787829666]  req@ffff8cd0c8e9de00 x1874671626962048/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829691 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:0.0:1787829666.902617:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3552
00000100:00080000:0.0:1787829666.902629:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829666.902641:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829666.902642:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626962048/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829691 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000008:00100000:0.0:1787829666.902657:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00080000:1.0:1787829666.902662:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00100000:0.0:1787829666.902675:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626962432/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:1.0:1787829666.902679:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829666.902686:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626962048:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829666.902728:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626962176:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829666.902753:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829666.902763:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829666/real 1787829666]  req@ffff8cd0c8e9cf00 x1874671626962176/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829691 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829666.902772:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626962176/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829691 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829666.902778:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829666.902780:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829666.902782:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626962176:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787829666.902787:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626962304:192.168.203.145@tcp:8:kworker.0
00000800:00000100:1.0:1787829666.902816:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829666.902824:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829666/real 1787829666]  req@ffff8cd0c8e9fc00 x1874671626962304/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829691 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829666.902832:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626962304/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829691 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829666.902838:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829666.902841:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829666.902843:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626962304:192.168.203.145@tcp:8:kworker.0
00000100:00100000:1.0:1787829666.902847:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626962432:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829666.902868:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829666.902876:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829666/real 1787829666]  req@ffff8cd0c8e9ed00 x1874671626962432/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829691 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829666.902884:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626962432/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829691 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829666.902890:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829666.902892:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829666.902894:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626962432:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829667.094199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829667.478255:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829668.118183:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829668.502228:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829669.142172:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829669.526222:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829670.166204:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829670.550257:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829671.190215:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829671.574261:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787829672.022180:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829672.022199:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787829672.022203:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829672.022212:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3557
00000100:00080000:0.0:1787829672.022221:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829672.022227:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829672.022271:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e580 x1874671626962560/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829672.022323:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829672.022326:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829672.022327:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829672.022330:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3557
00000100:00080000:0.0:1787829672.022334:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829672.022337:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829672.022345:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829672.022357:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9ed00 x1874671626962688/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1787829672.022395:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626962560:192.168.203.145@tcp:250:kworker.0
00000100:00080000:0.0:1787829672.022397:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829672.022408:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787829672.022411:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829672.022416:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3557
00000100:00080000:0.0:1787829672.022419:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829672.022424:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829672.022447:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829672.022457:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9fc00 x1874671626962816/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829672.022465:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829672.022468:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829672.022470:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829672.022472:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3557
00000100:00080000:0.0:1787829672.022494:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829672.022499:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829672.022506:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:1.0:1787829672.022506:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829672.022528:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9cf00 x1874671626962944/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1787829672.022581:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829672/real 1787829672]  req@ffff8cd0c8e9e580 x1874671626962560/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829697 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829672.022602:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e580 x1874671626962560/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829697 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829672.022624:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829672.022629:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829672.022634:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626962560:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829672.022657:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626962688:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829672.022700:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829672.022717:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829672/real 1787829672]  req@ffff8cd0c8e9ed00 x1874671626962688/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829697 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829672.022761:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9ed00 x1874671626962688/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829697 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829672.022768:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829672.022770:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829672.022773:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626962688:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787829672.022778:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626962816:192.168.203.145@tcp:8:kworker.0
00000800:00000100:1.0:1787829672.022819:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829672.022831:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829672/real 1787829672]  req@ffff8cd0c8e9fc00 x1874671626962816/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829697 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829672.022843:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9fc00 x1874671626962816/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829697 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829672.022851:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829672.022853:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829672.022856:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626962816:192.168.203.145@tcp:8:kworker.0
00000100:00100000:1.0:1787829672.022883:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626962944:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829672.022932:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829672.022941:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829672/real 1787829672]  req@ffff8cd0c8e9cf00 x1874671626962944/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829697 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829672.022949:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9cf00 x1874671626962944/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829697 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829672.022973:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829672.022976:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829672.022978:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626962944:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829672.214141:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829672.598218:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829673.238223:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829673.622219:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829674.262206:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829674.646250:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829675.286226:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829675.670297:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829676.310170:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829676.694242:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1787829677.142217:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829677.142229:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1787829677.142234:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829677.142245:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3562
00000100:00080000:0.0:1787829677.142258:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829677.142265:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:0.0:1787829677.142310:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9de00 x1874671626963072/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829677.142337:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829677.142340:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829677.142342:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829677.142344:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3562
00000100:00080000:0.0:1787829677.142347:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829677.142349:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829677.142359:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829677.142386:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626963072:192.168.203.145@tcp:250:kworker.0
00000100:00100000:0.0:1787829677.142391:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9e940 x1874671626963200/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829677.142400:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829677.142403:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:0.0:1787829677.142405:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829677.142408:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3562
00000100:00080000:0.0:1787829677.142411:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829677.142413:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829677.142417:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1787829677.142428:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9c3c0 x1874671626963328/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1787829677.142435:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1787829677.142438:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1787829677.142439:0:36:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1787829677.142441:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3562
00000100:00080000:0.0:1787829677.142451:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1787829677.142454:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:0.0:1787829677.142457:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000800:00000100:1.0:1787829677.142463:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:0.0:1787829677.142468:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8e9d680 x1874671626963456/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1787829677.142512:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829677/real 1787829677]  req@ffff8cd0c8e9de00 x1874671626963072/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829702 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829677.142528:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9de00 x1874671626963072/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829702 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829677.142537:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829677.142541:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829677.142545:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626963072:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829677.142575:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626963200:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829677.142602:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829677.142614:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829677/real 1787829677]  req@ffff8cd0c8e9e940 x1874671626963200/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829702 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829677.142623:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9e940 x1874671626963200/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829702 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829677.142629:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829677.142631:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829677.142634:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626963200:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787829677.142645:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626963328:192.168.203.145@tcp:8:kworker.0
00000800:00000100:1.0:1787829677.142662:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829677.142669:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829677/real 1787829677]  req@ffff8cd0c8e9c3c0 x1874671626963328/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829702 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829677.142676:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9c3c0 x1874671626963328/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829702 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829677.142682:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829677.142685:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829677.142688:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626963328:192.168.203.145@tcp:8:kworker.0
00000100:00100000:1.0:1787829677.142694:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626963456:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829677.142713:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829677.142725:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829677/real 1787829677]  req@ffff8cd0c8e9d680 x1874671626963456/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829702 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829677.142738:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8e9d680 x1874671626963456/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829702 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829677.142744:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829677.142747:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829677.142749:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8e9d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626963456:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829677.334214:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829677.718233:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829678.358232:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829678.742246:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829679.382159:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829679.766176:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829680.406191:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829680.790237:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829681.430157:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829681.814241:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787829682.262193:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829682.262205:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787829682.262210:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829682.262221:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3568
00000100:00080000:3.0:1787829682.262232:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829682.262241:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787829682.262293:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf840 x1874671626963584/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829682.262322:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829682.262325:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787829682.262327:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829682.262329:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3568
00000100:00080000:3.0:1787829682.262348:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829682.262352:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829682.262363:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1787829682.262369:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626963584:192.168.203.145@tcp:250:kworker.0
00000100:00100000:3.0:1787829682.262382:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eacf00 x1874671626963712/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829682.262397:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829682.262402:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787829682.262404:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829682.262409:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3568
00000100:00080000:3.0:1787829682.262414:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829682.262419:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829682.262424:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829682.262438:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8ead2c0 x1874671626963840/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1787829682.262446:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:3.0:1787829682.262452:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829682.262456:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787829682.262458:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829682.262463:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3568
00000100:00080000:3.0:1787829682.262466:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829682.262470:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829682.262473:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829682.262482:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae1c0 x1874671626963968/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1787829682.262495:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829682/real 1787829682]  req@ffff8cd0c8eaf840 x1874671626963584/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829707 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829682.262521:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf840 x1874671626963584/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829707 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829682.262531:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829682.262536:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829682.262541:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626963584:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829682.262572:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626963712:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829682.262598:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829682.262609:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829682/real 1787829682]  req@ffff8cd0c8eacf00 x1874671626963712/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829707 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829682.262618:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eacf00 x1874671626963712/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829707 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829682.262624:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829682.262626:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829682.262628:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eacf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626963712:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787829682.262632:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626963840:192.168.203.145@tcp:8:kworker.0
00000800:00000100:1.0:1787829682.262645:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829682.262651:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829682/real 1787829682]  req@ffff8cd0c8ead2c0 x1874671626963840/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829707 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829682.262658:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8ead2c0 x1874671626963840/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829707 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829682.262673:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829682.262675:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829682.262678:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8ead2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626963840:192.168.203.145@tcp:8:kworker.0
00000100:00100000:1.0:1787829682.262682:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626963968:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829682.262695:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829682.262702:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829682/real 1787829682]  req@ffff8cd0c8eae1c0 x1874671626963968/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829707 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829682.262709:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae1c0 x1874671626963968/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829707 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829682.262714:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829682.262715:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829682.262717:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626963968:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829682.454215:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829682.838283:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829683.478215:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829683.862279:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829684.502226:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829684.886279:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829685.526177:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829685.910314:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829686.550227:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829686.934246:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829687.382178:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829687.382189:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829687.382220:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829687.382231:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3573
00000100:00080000:2.0:1787829687.382243:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829687.382250:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829687.382296:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa940 x1874671626964096/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829687.382332:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829687.382335:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829687.382336:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829687.382339:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3573
00000100:00080000:2.0:1787829687.382343:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829687.382346:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829687.382357:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829687.382369:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8780 x1874671626964224/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829687.382377:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829687.382380:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829687.382384:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00100000:1.0:1787829687.382385:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626964096:192.168.203.145@tcp:250:kworker.0
00000100:00080000:2.0:1787829687.382389:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3573
00000100:00080000:2.0:1787829687.382395:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829687.382398:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829687.382401:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829687.382411:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da9a40 x1874671626964352/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829687.382428:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829687.382431:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829687.382432:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829687.382436:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3573
00000100:00080000:2.0:1787829687.382439:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829687.382442:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829687.382445:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829687.382454:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab0c0 x1874671626964480/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1787829687.382465:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829687.382525:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829687/real 1787829687]  req@ffff8cd0c3daa940 x1874671626964096/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829712 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829687.382541:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa940 x1874671626964096/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829712 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829687.382567:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829687.382570:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829687.382573:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626964096:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829687.382602:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626964224:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829687.382622:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829687.382631:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829687/real 1787829687]  req@ffff8cd0c3da8780 x1874671626964224/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829712 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829687.382639:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8780 x1874671626964224/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829712 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829687.382645:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829687.382648:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829687.382650:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626964224:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787829687.382654:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626964352:192.168.203.145@tcp:8:kworker.0
00000800:00000100:1.0:1787829687.382666:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829687.382672:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829687/real 1787829687]  req@ffff8cd0c3da9a40 x1874671626964352/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829712 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829687.382680:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da9a40 x1874671626964352/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829712 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829687.382685:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829687.382687:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829687.382689:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626964352:192.168.203.145@tcp:8:kworker.0
00000100:00100000:1.0:1787829687.382692:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626964480:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829687.382703:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829687.382708:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829687/real 1787829687]  req@ffff8cd0c3dab0c0 x1874671626964480/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829712 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829687.382721:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab0c0 x1874671626964480/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829712 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829687.382727:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829687.382729:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829687.382731:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626964480:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829687.574210:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829687.958250:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829688.598199:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829688.982230:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829689.622224:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829690.006297:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829690.646238:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829691.030184:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829691.670193:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829692.054257:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787829692.502260:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829692.502283:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787829692.502288:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829692.502303:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3578
00000100:00080000:3.0:1787829692.502316:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829692.502324:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787829692.502387:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eae940 x1874671626964608/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829692.502422:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829692.502426:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787829692.502427:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829692.502431:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3578
00000100:00080000:3.0:1787829692.502434:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829692.502437:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829692.502448:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829692.502493:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac000 x1874671626964736/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829692.502502:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829692.502505:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:3.0:1787829692.502507:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829692.502510:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3578
00000100:00080000:3.0:1787829692.502513:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00100000:1.0:1787829692.502518:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626964608:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787829692.502519:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829692.502529:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829692.502538:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626964864/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829692.502546:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829692.502549:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787829692.502550:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829692.502553:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3578
00000100:00080000:3.0:1787829692.502556:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829692.502560:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829692.502575:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829692.502584:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626964992/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1787829692.502622:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829692.502683:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829692/real 1787829692]  req@ffff8cd0c8eae940 x1874671626964608/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829717 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829692.502698:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eae940 x1874671626964608/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829717 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829692.502708:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829692.502712:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829692.502715:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626964608:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829692.502735:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626964736:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829692.502756:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829692.502766:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829692/real 1787829692]  req@ffff8cd0c8eac000 x1874671626964736/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829717 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829692.502785:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac000 x1874671626964736/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829717 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829692.502802:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829692.502804:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829692.502807:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626964736:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787829692.502832:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626964864:192.168.203.145@tcp:8:kworker.0
00000800:00000100:1.0:1787829692.502849:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829692.502856:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829692/real 1787829692]  req@ffff8cd0c8eada40 x1874671626964864/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829717 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829692.502868:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626964864/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829717 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829692.502877:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829692.502881:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829692.502884:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626964864:192.168.203.145@tcp:8:kworker.0
00000100:00100000:1.0:1787829692.502890:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626964992:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829692.502911:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829692.502925:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829692/real 1787829692]  req@ffff8cd0c8eac780 x1874671626964992/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829717 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829692.502937:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626964992/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829717 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829692.502943:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829692.502945:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829692.502953:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626964992:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829692.694212:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829693.078245:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829693.718179:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829694.102271:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829694.742242:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829695.126232:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829695.766167:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829696.150219:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829696.790231:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829697.174219:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:3.0:1787829697.622194:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829697.622204:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:3.0:1787829697.622209:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829697.622219:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3583
00000100:00080000:3.0:1787829697.622233:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829697.622243:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:3.0:1787829697.622295:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac3c0 x1874671626965120/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829697.622326:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829697.622330:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787829697.622332:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829697.622336:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3583
00000100:00080000:3.0:1787829697.622340:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829697.622345:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829697.622358:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829697.622373:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eaf0c0 x1874671626965248/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829697.622383:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829697.622387:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00100000:1.0:1787829697.622388:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626965120:192.168.203.145@tcp:250:kworker.0
00000100:00080000:3.0:1787829697.622389:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829697.622397:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3583
00000100:00080000:3.0:1787829697.622400:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829697.622403:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829697.622427:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829697.622438:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eac780 x1874671626965376/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:3.0:1787829697.622445:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:3.0:1787829697.622448:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:3.0:1787829697.622449:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:3.0:1787829697.622452:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3583
00000100:00080000:3.0:1787829697.622454:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:3.0:1787829697.622457:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:3.0:1787829697.622460:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:3.0:1787829697.622467:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c8eada40 x1874671626965504/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1787829697.622479:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829697.622528:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829697/real 1787829697]  req@ffff8cd0c8eac3c0 x1874671626965120/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829722 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829697.622544:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac3c0 x1874671626965120/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829722 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829697.622553:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829697.622556:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829697.622559:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626965120:192.168.203.145@tcp:250:kworker.0
00000100:00100000:1.0:1787829697.622584:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626965248:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829697.622601:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829697.622610:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829697/real 1787829697]  req@ffff8cd0c8eaf0c0 x1874671626965248/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829722 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829697.622619:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eaf0c0 x1874671626965248/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829722 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829697.622624:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829697.622626:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829697.622628:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eaf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626965248:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787829697.622632:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626965376:192.168.203.145@tcp:8:kworker.0
00000800:00000100:1.0:1787829697.622645:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829697.622660:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829697/real 1787829697]  req@ffff8cd0c8eac780 x1874671626965376/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829722 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829697.622668:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eac780 x1874671626965376/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829722 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829697.622674:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829697.622676:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829697.622678:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626965376:192.168.203.145@tcp:8:kworker.0
00000100:00100000:1.0:1787829697.622682:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626965504:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829697.622718:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829697.622726:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829697/real 1787829697]  req@ffff8cd0c8eada40 x1874671626965504/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829722 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829697.622734:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c8eada40 x1874671626965504/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829722 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829697.622740:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829697.622742:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829697.622744:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c8eada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626965504:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829697.814165:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829698.198220:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829698.838154:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829699.222212:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829699.862170:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829700.246206:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829700.886130:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829701.270183:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829701.910157:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829702.294200:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1787829702.742297:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) f77eba79-b913-4677-9c7e-d487547b7ee2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829702.742314:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1787829702.742321:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829702.742337:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.145@tcp: connect to NID 192.168.203.145@tcp last attempt 3588
00000100:00080000:2.0:1787829702.742354:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.145@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829702.742368:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.145@tcp: import ffff8cd0c36c4000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:2.0:1787829702.742430:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dabc00 x1874671626965632/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829702.742502:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829702.742509:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829702.742513:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829702.742522:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3588
00000100:00080000:2.0:1787829702.742534:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829702.742541:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0d27ee000: import ffff8cd0d23bc800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829702.742570:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626965632:192.168.203.145@tcp:250:kworker.0
00000008:00100000:2.0:1787829702.742579:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829702.742605:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3daa580 x1874671626965760/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1787829702.742629:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) c8f9ccc5-49e2-4db1-a0cd-da1b75937409->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1787829702.742637:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery
00000100:00080000:2.0:1787829702.742641:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829702.742673:0:38:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: connect to NID 192.168.203.145@tcp last attempt 3588
00000800:00000100:1.0:1787829702.742683:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00080000:2.0:1787829702.742690:0:38:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1787829702.742704:0:38:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff8cd0d27ee000: import ffff8cd0d23b8000 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000008:00100000:2.0:1787829702.742719:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829702.742748:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3dab840 x1874671626965888/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1787829702.742755:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829702/real 1787829702]  req@ffff8cd0c3dabc00 x1874671626965632/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829727 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829702.742809:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00100000:1.0:1787829702.742816:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dabc00 x1874671626965632/t0(0) o250->MGC192.168.203.145@tcp@192.168.203.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829727 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:2.0:1787829702.742820:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1787829702.742827:0:38:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1787829702.742843:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: connect to NID 192.168.203.145@tcp last attempt 3588
00000100:00080000:1.0:1787829702.742854:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0c36c4000 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:2.0:1787829702.742856:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: tried all connections, increasing latency to 20s
00000100:00080000:1.0:1787829702.742868:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.145@tcp failed (-110)
00000100:00080000:2.0:1787829702.742875:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8cd0c360a800: import ffff8cd0d225d800 using connection 192.168.203.145@tcp/192.168.203.145@tcp
00000100:00100000:1.0:1787829702.742887:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dabc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f77eba79-b913-4677-9c7e-d487547b7ee2:2355:1874671626965632:192.168.203.145@tcp:250:kworker.0
00000008:00100000:2.0:1787829702.742895:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1787829702.742925:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff8cd0c3da8000 x1874671626966016/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1787829702.742940:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626965760:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829702.742987:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829702.743005:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829702/real 1787829702]  req@ffff8cd0c3daa580 x1874671626965760/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829727 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829702.743021:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3daa580 x1874671626965760/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0d27ee000@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829727 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829702.743031:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23bc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829702.743036:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829702.743039:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3daa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626965760:192.168.203.145@tcp:38:kworker.0
00000100:00100000:1.0:1787829702.743056:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626965888:192.168.203.145@tcp:8:kworker.0
00000800:00000100:1.0:1787829702.743079:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829702.743131:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829702/real 1787829702]  req@ffff8cd0c3dab840 x1874671626965888/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829727 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829702.743140:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3dab840 x1874671626965888/t0(0) o8->lustre-OST0001-osc-ffff8cd0d27ee000@192.168.203.145@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829727 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829702.743147:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d23b8000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829702.743149:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829702.743151:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3dab840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c8f9ccc5-49e2-4db1-a0cd-da1b75937409:2355:1874671626965888:192.168.203.145@tcp:8:kworker.0
00000100:00100000:1.0:1787829702.743178:0:2355:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626966016:192.168.203.145@tcp:38:kworker.0
00000800:00000100:1.0:1787829702.743197:0:2355:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.145@tcp
00000100:00100000:1.0:1787829702.743205:0:2355:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829702/real 1787829702]  req@ffff8cd0c3da8000 x1874671626966016/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829727 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1787829702.743214:0:2355:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff8cd0c3da8000 x1874671626966016/t0(0) o38->lustre-MDT0000-mdc-ffff8cd0c360a800@192.168.203.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829727 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1787829702.743219:0:2355:0:(import.c:65:import_set_state_nolock()) ffff8cd0d225d800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1787829702.743222:0:2355:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.145@tcp failed (-110)
00000100:00100000:1.0:1787829702.743224:0:2355:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8cd0c3da8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5e8a6537-b6d4-4fd4-a0a9-f7ceeb471f6f:2355:1874671626966016:192.168.203.145@tcp:38:kworker.0
00000100:00100000:3.0:1787829702.934140:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829703.318199:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829703.958198:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829704.342177:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829704.982192:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829705.366212:0:5595:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00100000:3.0:1787829706.006225:0:10289:0:(client.c:2715:ptlrpc_set_wait()) set ffff8cd0c3dad180 going to sleep for 0 seconds
00010000:00010000:0.0:1787829706.390200:0:5594:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
Debug log: 43542 lines, 43542 kept, 0 dropped, 0 bad.